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Details of the Approach to Conducting the Adult 
Education Strategies Evidence Review 
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This appendix contains the technical details of the approach followed for the systematic evidence 
review of research on the effectiveness of strategies in adult education. It includes more information 
about the organizing framework for the review and describes the process for defining its scope, 
identifying relevant literature, screening references and reviewing studies, and summarizing and 
reporting on findings. Details are provided so that someone familiar with systematic evidence reviews 
can understand the approach that was used and how it draws on the approach specified by the 
Institute of Education Sciences’ What Works Clearinghouse (WWC). Some technical terms are 
explained throughout for readers less familiar with the WWC or research design. Exhibit A.1 presents 
an overview of the approach, followed by a more detailed and technical presentation of how the review 
was conducted. 


Exhibit A.1. Overview of the approach for conducting the systematic evidence review 


How were studies identified? 

e Bysearching a large set of electronic databases and websites—including those identified by the Institute of 
Education Sciences’ What Works Clearinghouse (WWC) and experts in the field—for publicly available 
literature. 

By using a broad set of search terms for the eligible populations and allowable activities under Title II of 
the Workforce Innovation and Opportunity Act (WIOA). 


How were studies determined to be eligible for review? 
Studies had to meet each of the following criteria: 


e Type of learners: Those who were age 16 or older who were not enrolled in high school, and who met at 

least one of the following three criteria for adults eligible for federally-funded adult education: 

— Lacked basic skills, such as literacy and numeracy 

— Lacked English language proficiency 

— Did not have a high school diploma or a high school equivalency credential 
Strategy being evaluated: At least one allowable under Title II, including, but not limited to, adult 
education and literacy activities, integrated education and training, English language acquisition activities, 
and workforce preparation activities. Studies testing some strategies were excluded: (1) Family literacy 
services, due to resource limitations and WIOA’s primary focus on workforce development, and (2) Title 
II-eligible strategies when combined with non-eligible, large-scale services, such as residential programs, 
because the effect of eligible activities could not be isolated. 


Location and timing: Conducted in the United States and published within the past 20 years. This allows 
for the inclusion of research that adequately represents the status of the field and avoids the inclusion of 
research conducted with populations and in contexts that may be very different from those in which Title 
II services are implemented. 

Outcomes: Examine at least one outcome aligned with the performance indicators for programs receiving 
Title II funding. These indicators fall into four areas: academic skills, educational progress, employment, 
and earnings. 

Research design: Use a randomized controlled trial, quasi-experimental, regression discontinuity, or 
single-case design. These are all methods that can provide evidence that effects are due to the strategies 
tested and for which WWC standards and procedures can be applied. 


How were the eligible studies reviewed? 

e By following the WWC’s version 4.0 Standards and Procedures Handbooks and the Review of Individual 
Studies Protocol (RISP) version 4.0.! This allowed the study team to identify studies that used a rigorous 
design to examine effectiveness. 

How are findings reported? 

e By grouping outcomes into areas that align with the primary indicators of performance for programs 
receiving Title II funding—academic skills, educational progress, employment, and earnings. 
Characterized as having either a positive effect, negative effect, no effect, or a mixed effect (that is, a 
combination of statistically significant effects that were both positive and negative). 


‘The WWC version 4.0 Standards and Procedures Handbooks can be found at: littps://ies.ed.gov/ncee/wwe/Handbooks. The 
WWC’s Review of Individual Studies Protocol can be found at: https://ies.ed.gov/ncee/wwe/Handbooks#protocol. 
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A.1. Applying a Framework to the Systematic Evidence Review 


To help summarize the evidence about adult education strategies, reviewed studies and their findings 
were organized into high-level groups or categories (Exhibit A.1). These categories were developed in 
consultation with researchers, practitioners, and policymakers. 


Define the purpose: Identify evidence of effective strategies in adult education and 
identify gaps in the research. 


This review of existing research responds to a mandate from Title II of the Workforce Innovation and 
Opportunity Act of 2014 (WIOA), also known as the Adult Education and Family Literacy Act. This 
mandate specifies that ED conduct a national assessment of adult education, including examining the 
extent to which strategies in adult education are effective. A necessary component of the review was 
identifying rigorous studies that tested the impact of strategies in adult education on key learner 
outcomes in the following areas: academic skills, educational progress, employment, and earnings. 
These areas are closely tied to the goals of adult education as emphasized under Title II and directly 
relate to the outcomes for which federally supported adult education programs are held accountable 
under WIOA. 


Determine and use a clear process: Apply the What Works Clearinghouse standards 
and procedures. 


The review, described in greater detail below, was guided by specifications set forth by the WWC in its 
version 4.0 Standards and Procedures Handbooks. Following WWC standards and procedures ensures 
that this review used an objective and consistent process for assessing the quality of the research 
evidence. The WWC was developed to identify high quality research that answers the question, “What 
works in education?”. To do so, the WWC sets standards and procedures to identify well-designed and 
well-implemented effectiveness studies. These studies provide trustworthy evidence that the effects 
observed are the result of the strategies tested rather than other potential sources. 


Organize adult education strategies into seven categories. 


Table A.1, which was developed in consultation with researchers, practitioners, and policymakers lists 
the adult education categories and some corresponding examples of strategies that might be tested. 


e The categories are meant to be distinct from one another. For example, the literacy instruction 
and English language acquisition instruction categories both involve literacy, but the English 
language acquisition category captures instruction that specifically targets learners whose native 
language is not English and who lack English proficiency. The career development and transition 
instruction category and the individualized and targeted supports for learners category both focus on 
supporting learners but the career development and transition instruction includes programs like 
Bridge classes and IET while individualized and targeted supports include supplemental supports. 


e The strategies include some examples that are emphasized in WIOA, in wide use in the field, or 
innovative and expected to be of growing interest. The strategies within categories are not intended 
to be comprehensive, but rather illustrative examples of strategies that could be tested and might 
contribute to an understanding of the category. 
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The review team used the authors’ descriptions of what was examined in the study to identify the 
relevant strategies and subsequently the categories of adult education that were examined. Studies may 
have examined strategies in more than one category. Reviewers did not code the quality of 
implementation of the strategies. Authors were not asked to categorize their own studies or confirm 
the review team’s categories, but content experts were consulted, when necessary. This means that the 
research team may have identified strategies and categorized studies in ways that differ from how 
others may have labeled the same studies. 


Table A.1. Categories and example strategies in adult education 


Categories Strategies 
Literacy instruction 


Definition 


Explicit instruction on reading 


Direct and purposeful teaching focused on developing the components of 
reading. This could include decoding (such as the ability to apply 
knowledge of letter-sound relationships to correctly pronounce printed 
words); fluency (such as the ability to read with speed and accuracy); 
vocabulary (such as the depth, breadth, and flexibility of knowledge 
about words); and comprehension (such as understanding language from 
a written code that represents concepts and communicates information 
and ideas). 


Extended reading practice 


Instruction that provides students with reading practice (for example, 
engaging with reading materials that are personally interesting or 
embedding reading practice in meaningful learning activities) that allows 
for application of new skills. 


Explicit framing 


Instructional practice that provides a rationale for what is to be learned, 
the activities to be completed, and how they will help the learner. 


Explicit writing instruction 


Direct and purposeful teaching focused on developing writing skills. This 
could include brainstorming; drafting (including selecting the right words 
and syntactic structure to convey the intended meaning, sentence 
construction, spelling, handwriting, keyboarding, capitalization, 
punctuation); getting feedback; revising; editing; and helping adults feel 
comfortable with writing, find their voice, develop an identify as a writer, 
understand how writers write, explore self-expression, and enhance 
communication. 


Integrated reading and writing instruction 


Instruction that integrates reading and writing—for example, by having 
students read some text and then produce a series of essays or responses 
to topics from that text. 
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Categories Strategies 


Instruction on spelling 


Definition 


Instruction that focuses on teaching students how to spell. This could 
include auditory approaches like phonics (for example, knowing the 
sounds of letters, breaking up words into syllables, and recognizing 
patterns); visual approaches (for example, highlighting difficult parts of 
words, looking for smaller words within words); kinesthetic approaches 
(for example, tracing letters, using Cloze exercises, rewriting incorrect 
words, using words in writing); and mnemonic approaches (such as 
memory aids). 


Literacy curricular materials designed for instruction 


Specific literacy curricular materials designed for instruction (for 
example, textbooks, modules, software, and other branded materials). 


Real-world contexts in instruction, which can include authentic 
activities and materials 


Instruction that uses authentic activities and materials not created for 
classroom instruction, gathered from actual contexts in which adults use 
the materials (for example, a workplace or restaurant) with the goal of 
improving literacy. 


Numeracy instruction 


Constructivist teaching 


A student-centered and active learning approach to instruction; a 
program/curriculum/pedagogy in which students (rather than the 
instructor) are guided so they are the ones who make sense of the 
mathematics, connect ideas, conjecture, describe patterns, and 
generalize from them. This contrasts with explicit instruction (also known 
as “lecture” or “demonstration” approaches). 


Instruction focused on multiple strands of mathematical 
proficiency 


A program/curriculum/pedagogy that focuses on more than one of the 

five strands of mathematical proficiency: 

e Conceptual understanding: Comprehension of mathematical 
concepts, operations, and relations 

e Adaptive reasoning: Capacity for logical thought, reflection, 
explanation, and justification 

e Strategic competence: Ability to formulate, represent, and solve 
mathematical problems 

e Procedural fluency: Skill in carrying out procedures flexibly, 
accurately, efficiently, and appropriately 

e Productive disposition: Habitual inclination to see mathematics as 
sensible, useful, and worthwhile, coupled with a belief in diligence 
and one’s own efficacy 
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Categories Strategies 


Numeracy curricular materials designed for instruction 


Definition 


Specific numeracy curricular materials designed for instruction (for 
example, textbooks, modules, software, and other branded materials). 


Real-world contexts in instruction 


Mathematical practices, including problem solving and 
mathematical justification 


Instruction that uses authentic activities and materials not created for 
classroom instruction, gathered from the actual contexts in which adults 
use those materials (for example, a workplace or restaurant) with the goal 
of improving numeracy. 

Instruction that provides students with opportunities to solve problems 
and then provides them with feedback. 


English language acquisition (ELA) instruction 


Focus on listening and speaking 


ELA/English as a second language (ESL) instruction that focuses on the 
listening and speaking components of literacy, including pronunciation. 


Focus on reading and writing 


ELA instruction that focuses on the reading and writing components of 
literacy. 


Integrate civics education into ELA instruction 


Instruction in English language acquisition and instruction on the rights 
and responsibilities of citizenship and civic participation. 


ELA curricular materials designed for instruction 


Specific ELA curricular materials designed for instruction (for example, 
textbooks, modules, software, and other branded materials). 


Real-world contexts in instruction, which can include authentic 
activities and materials 


Instruction that uses authentic activities and materials not created for 
classroom instruction, gathered from the actual contexts in which adults 
use those materials (for example, a workplace or restaurant) with the goal 
of improving English language proficiency. 


Focus on language and terminology, also referred to as 
functional language skills, needed for the workplace 


ELA instruction on the language and terminology relevant for specific 
occupational fields and industry certification exams in those fields. 


Connect native language literacy skills and learning strategies to 
English acquisition 
Adult secondary education (ASE) 


Provide opportunities for learners literate in their native languages to 
transfer their literacy and learning strategies to the acquisition of English. 


Preparation for high school (HS) equivalency test 
Competency-based HS equivalency 


Instruction focused on preparing a learner to take an HS equivalence test. 
An approach that allows learners to progress toward demonstrating 
mastery of HS academic content, regardless of time, place, or pace of 
learning. 


ASE curricular materials or models designed for instruction 


Specific ASE curricular materials designed for instruction (for example, 
textbooks, modules, software, and other branded materials). 
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Categories Strategies 


Real-world contexts in instruction, which can include authentic 
activities and materials 


Definition 


Instruction that uses authentic activities and materials gathered from the 
actual contexts in which adults use those materials (for example, a 
workplace or restaurant) with the goal of helping learners obtain a high 
school degree or the equivalent. 


Tools to improve access to instruction 


Distance learning 


Blended learning 


An approach that delivers all instruction in an entirely virtual format. The 
technology used can include the Internet, broadcast media, closed circuit 
or cable television, wireless communication devices, videos, DVDs, or CD- 
ROMs. 


An approach that delivers instruction through a combination of in-person 
and virtual instruction, with components integrated and required as part 
of the course. 


Mobile or online learning tools to supplement instruction 


Resources that learners can independently access outside of the 
classroom or through the Internet on a computer or mobile device to 
reinforce classroom instruction. 


Technology learning tools or virtual learning environments 
accessed in the classroom with oversight from an instructor 


Approaches that use learning tools or virtual learning environments 
within a physical classroom with oversight from an instructor. 


Self-paced in-person facilitated learning 


Alternative locations and times to accommodate working 
schedules 


Approaches in which learners can move through the learning materials at 
their own pace with the support of a tutor or instructor. 


Programs offered at alternative locations (for example, at job sites or 
libraries) with flexible or convenient times to allow learners to plan 
around work schedules. 


Intake and orientation models 


Attendance policies 


Approaches to learner intake and orientation. An intake process allows 
staff to assess individuals’ academic skills and other factors that could 
affect their ability to succeed in a program (for example, child care needs, 
motivation, goals, career interests, academic experiences, learning styles, 
and previous academic assessments). An orientation process includes an 
introduction to and overview of the program and criteria for completing 
it, student responsibilities, any attendance policies, and counseling 
opportunities. 

Programs that enforce an attendance policy requiring learners to 


participate in a number or percentage of classes/activities to stay 
enrolled. 
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Categories Strategies 


Employer partnerships; for example, employer based or 
employer provided 


Definition 


Programs either based with or provided by an employer. 


Career development and transition instruction 


Integrated education and training (IET) using co-teaching 
approaches 


IET through co-planning and alternate teaching 


Programs in which two teachers—one basic skills teacher and one 
occupational skills teacher—teach integrated curricula (adult education 
instruction with workforce preparation activities and workforce training) 
simultaneously and focused on a single set of learning goals. The model 
involves teachers planning together and teaching in the same classroom 
at the same time. 


Programs in which two teachers—one basic skills teacher and one 
occupational skills teacher—teach integrated curricula (adult education 
instruction with workforce preparation activities and workforce training) 
simultaneously and focused on a single set of learning goals. The model 
involves co-planning and alternate teaching, in which teachers plan 
together but teach at separate class times. 


Contextualized basic skills instruction in literacy, numeracy, and 
English language acquisition 


Programs with a group of instructional strategies designed to link the 
learning of basic skills with academic or occupational content by focusing 
teaching and learning directly on concrete applications in a specific 
career context of interest to students. This can include vocational ESL 
programs. 


Coordinated enrollment in adult education and occupational 
skills training 


Programs in which students are enrolled in both adult education and 
occupational skills programs that do not share a curriculum. 


Pre-apprenticeships 


Programs or strategies designed to prepare individuals to enter and 
succeed in a registered apprenticeship. Components of pre- 
apprenticeships may include the following: a partnership with a 
registered apprentice sponsor, an approved training and curriculum, 
hands-on training, industry-recognized credentials, and supportive 
services. 


Alternative credentialing or badges 


Programs that enable students to learn from non-degree coursework to 
improve their skills and employability. This learning may culminate in 
digital badges, certificates, and micro-credentials. 
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Categories Strategies 


On-ramp and bridge classes 


Definition 


Classes or programs to prepare learners to transition to either 
occupational training by providing basic skill instruction along with 
occupational content or employment skills instruction or to college 
courses by providing basic skill instruction along with college success 
strategies. 


Co-enrollment with developmental education at colleges 


Approaches in which learners co-enroll in adult education programs and 
nontransferable courses, such as developmental education or “college 
success” courses (which teach students how to navigate through college 
life, including teaching them study skills, how to access various student 
resources, and so forth) and transferable, credit-bearing classes, often in 
specific industries or career fields. 


Co-enrollment with credit courses at colleges 


Approaches in which learners co-enroll in adult education programs and 
courses that provide transferable, college-level credit. Such programs 
allow students the opportunity to develop skills and gain credentials 
within their specific career field while working on their secondary 
credential. 


Workforce preparation 


Instruction that focuses on helping students learn skills necessary to be 
successful in the workforce (for example, critical thinking, digital literacy, 
self-management, and employability). 


Individualized and targeted learner supports 


Career navigation supports 


Wraparound services to address basic needs and logistical 
barriers 


Supports designed to help learners progress on a career path. Supports 
include career exploration and planning assistance; assistance in 
accessing financial supports for education; and support in developing 
study and work skills. 

Services to address basic needs and logistical barriers that learners might 
face (for example, basic needs, transportation, child care, housing, and 
supports). 


Texting or calls to encourage persistence 


Approaches to remind students/encourage persistence by using text 
messages or phone calls. 


Re-entry initiatives and post-release services for formerly 
incarcerated individuals 


Programs with a focus on helping formerly incarcerated individuals 
successfully transition to education and careers post-release. 


A-il 


A.2. Defining the Scope of the Review 


The initial search for existing research studies cast a wide net to identify all relevant adult education 
literature, but then focused in on the most relevant studies. 


Focus the scope on WIOA Title II-eligible activities, populations, and settings. 


Eligibility criteria were established to align with the eligible adult education activities and individuals 
under Title II. 


Type of learners 


To be eligible for review, at least 50 percent of participants in the study—called the “study sample”— 
needed to meet several criteria. A study must have been conducted with learners who were age 16 and 
older; not enrolled in high school; and who lacked basic skills (for example, literacy, numeracy), lacked 
English language proficiency, or had no high school diploma or equivalency. A study was also eligible 
for review if it reported on a subgroup, that is a subset of learners, in which all group members 
satisfied these criteria. 


Strategy being evaluated 


The focus of this review was on strategies allowable under Title II. Allowable activities include, but are 
not limited to, adult education and literacy activities, integrated education and training, English 
language acquisition activities, and workforce preparation activities. Due to limited resources and 
WIOA’s focus on workforce development, studies were excluded that examined family literacy 
services. In addition, studies in which WIOA Title II eligible activities were combined with non-eligible 
activities were excluded. For example, studies that provided sample members with education and 
training alongside free housing were excluded from review because it was not possible to isolate the 
effect of the WIOA-eligible education and training activities from the effect of the non-eligible 
residential component. While this review uses the term strategy to refer to what was tested in a study, 
the studies themselves often used the terms treatment or intervention, which are also the terms used by 
the WWC. 


Location and timing 


Only studies conducted in the United States and that were published within the last 20 years were 
included in this review. These are standard criteria that reflect a preference to include studies in 
contexts most similar to those eligible for Title II services. Labor markets in other countries may be 
sufficiently different from those in the United States, limiting the generalizability of findings from non- 
U.S. studies related to employment and earnings outcomes. Although non-U.S. studies could help to 
inform what works in the U.S. regarding outcomes such as academic skills, because the review applied 
a single protocol systematically to all studies and outcomes, non-U.S. studies were excluded from this 
review. 
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Outcomes 


Studies needed to examine an outcome--that is, a measure of learners’ knowledge, skills or behaviors 
expected to change as a result of the strategy-- aligned with WIOA primary indicators of performance, 
including for Title II programs. Accordingly, studies that measured academic skills, educational 
progress, employment, or earnings were eligible for inclusion in the review. 


Include only studies with research designs eligible under the What Works 


Clearinghouse. 


The WWC’s Procedures Handbook version 
4.0 was used to determine study eligibility. 
Specifically, studies were required to use 
research designs that can provide evidence 
that effects are due to the strategies tested. 
The WWC identifies four such designs— 
randomized controlled trial, quasi- 
experimental, regression discontinuity, or 
single-case designs. 


A.3. Searching for Relevant 
Literature on Strategies in 
Adult Education 


The review cast a wide net in its search for 
relevant published and unpublished 
literature. This effort included developing 
broad and inclusive search terms, searching 
a wide range of electronic databases and 
websites, and seeking recommendations 
from experts in the field of adult education. 


Use a broad set of search terms. 


The review used a broad set of search terms 
informed by the field (Table A.2). The 

selected terms describe the adult education- 
eligible populations and outcomes that align 


The WWC identifies four eligible research study designs for 
determining, with some confidence, whether something 
works: 


Randomized controlled trial (RCT): When individuals 
or clusters of individuals in a study are randomly assigned 
to either receive the strategy being tested or not receive 
the strategy through a process like a lottery or coin toss. 
The groups are then compared to determine if the strategy 
was effective at improving outcomes like skills, educational 
progress, employment, and earnings. 
Quasi-experimental design (QED): When decisions 
about which individuals or clusters of individuals in a 
study receive the strategy being tested are made using a 
process that is not random. The groups are then compared 
to determine if the strategy was effective at improving 
outcomes. 

Regression Discontinuity Design (RDD): When 
individuals are assigned to receive a strategy or not based 
on a specific threshold on a continuous scale (such as 
scoring above a cut-off number on an achievement test). 
The groups are then compared to determine if the strategy 
was effective at improving outcomes. 


Single-Case Design (SCD): When individuals receive the 
strategy being studied some, but not all, the time. 
Outcomes for the individual are measured at multiple 
times, both when they are receiving the strategy and not, 
to determine if the presence of the strategy improved their 
outcomes. 


Source: Adapted from the What Works Clearinghouse Glossary. 


https://ies.ed.gov/ncee/wwc/Glossary 


with the primary indicators of performance for programs receiving Title II funding—academic skills, 
educational progress, employment, and earnings. The search also included terms capturing evaluation, 
adult education strategies, and analysis concepts to identify studies that examined the effectiveness of 


strategies for adult education. 


Table A.2. Search terms used in literature search 


Category 


Purpose of 
including terms 


Search terms 


Population "Adult basic education*" OR "Adult basic learner*" OR "Adult basic student*" OR "GED 

Terms to identify student*" OR "GED instruction" OR "HiSET student*" OR "HiSET instruction" OR "TASC* 

references that report Student*" OR "TASC instruction" OR "High school equivalenc*" OR "HSE student*" OR 

on Title Il-eligible “Adult secondary*” OR "Adult English learner*" OR "Adult English student*" OR "Adult 

learners English language learner*" OR "Adult English language student*" OR "Adult English second 
language learner*" OR "Adult English second language student*" OR "Adult ELL*" OR "Adult 
ESL*" OR "Adult ESOL*" OR "Adult English for Speakers of Other Language*" OR "Adult 
literacy" OR "Low-literacy adult*" OR "Low* skilled adult*" OR "Adult* with limited 
education" OR "Adult* with limited literacy" OR "High school dropout*" OR "high school 
drop out*" OR "Early school leaver*" OR "Disconnected youth*" OR "Out-of-school youth*" 

Outcomes Employment OR Earnings OR "Labor market" OR Credential* OR "Carnegie unit*" OR 

Terms to identify "Course credit*" OR "education* credit*" OR "credit attain*" OR "high school diploma*" OR 

references that report “High school equivalen*" OR "Secondary school equivalen*" OR "Secondary school 

on measures that diploma*" OR "Grade level*" OR "Functioning level*" OR Transcript* OR "Report card*" OR 

align with primary "education* goal*" OR "education* milestone*" OR "education* exam*" OR Persistence OR 

indicators of "Staying in program" OR "Postsecondary transition" OR “Postsecondary enrollment” OR 

performance of Title "Basic skill*" OR "English language skill*" OR "Oral skill*" OR "Written skill*" OR "Academic 

II programs achievement" OR "academic attainment" OR Literacy OR Reading OR "Word reading" OR 
Fluency OR Vocabulary OR Comprehension OR Numeracy OR "Math* skill*" OR "math* 
achievement" OR "math* gain*" OR "Functional skill*" OR "Workplace skill*" OR 
"Occupational skill*" OR "Critical thinking" OR "Digital literacy" OR "Self-management" OR 
"OJT" OR "On the job training" OR apprentic* OR "trade benchmark*" OR "trade-related 
benchmark*" OR "measurable skill*" 

Strategy tested Curricul* OR program* OR intervention OR implement* OR demonstration* OR pilot* OR 

(Referred to as strateg* OR approach* OR practice OR model* OR policy OR policies OR experiment OR 

Intervention by initiative OR training 

WWC) 

Terms to identify 

references that report 

on strategies 

Evaluation analys* OR assess* OR compar* OR estimat* OR evaluat* OR evidence OR examin* OR 

Terms to identify "literature review" OR measur* OR "meta analys*" OR metaanalys* OR predict* OR 

references that report "Systematic review" OR varia* 

on whether strategies 

were assessed for 

effectiveness 

Analysis Affect* OR Benefit* OR Declin* OR Decreas* OR Develop* OR Effect* OR Efficac* OR Gain 

Terms to identify OR Growth OR Higher OR Increas* OR Impact* OR Improv* OR Lower OR Progress OR 

references that report Reduc* OR Statistic* OR Success* 

on whether strategies 

were effective 
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Search a large set of databases and websites to identify publicly available literature. 


In March 2019, searches were conducted for publicly available literature—in the electronic databases 
and websites listed in Table A.3. These sources include those listed in Appendix B of the WWC 
Procedures Handbook as well as additional sources identified by the field as being relevant to adult 
education. 


Table A.3 Databases and websites searched 


RYolines) Database or website 


Appendix B.2 of | Academic Search Premier, EconLit, Education Research Complete, E-Journals, ERIC, ProQuest 
WWC Procedures Dissertations & Theses, PsycINFO, SAGE Journals Online, Scopus, SocINDEX, WorldCat 
Handbook 


Appendix B.3 of — Abt Associates, Alliance for Excellent Education, American Education Research Association, 

WWC Procedures American Enterprise Institute, American Institutes of Research, Best Evidence Encyclopedia, 

Handbook Brookings Institution, Carnegie Corporation of New York, Center for Research and Reform in 
Education, Congressional Research Service, Government Accountability Office, 
Grants/contracts awarded by the Institute of Education Sciences, Heritage Foundation, Hoover 
Institution, Mathematica, MDRC, National Association of State Boards of Education, National 
Governors Association, Policy Archive, Policy Study Associates, RAND, Regional Educational 
Laboratories, SRI, Thomas B. Fordham Institute, Urban Institute 


Appendix B.4 of | Campbell Collaboration, Cochrane Central Register of Controlled Trials, Cochrane Database of 

WWC Procedures Systematic Reviews, Council for Learning Disabilities, Database of Abstracts of Reviews of 

Handbook Effects, Florida Center for Reading Research, Harvard Family Research Project, Institute for 
Higher Education Policy, Institute for Public Policy and Social Research, Learning Disabilities 
Association of America, Linguistic Society of America, Natl. Association for Bilingual 
Education, Natl. Association of State Directors of Career Tech. Ed., Natl. Association of State 
Directors of Special Education, Natl. Autism Center - National Standards Project, Natl. Center 
for Learning Disabilities, Natl. Center on Response to Intervention, Natl. Center on Secondary 
Education and Transition, Natl. College Access Network, Natl. Dropout Prevention 
Center/Network, National Bureau of Economic Research Working Papers, Teachers of English 
to Speakers of Other Languages 


Additional adult Adult Basic Education Teaching & Learning System (ATLAS), Barbara Bush Foundation, Center 


education for Applied Linguistics, Center for Study of Adult Literacy, Center for Law and Social Policy, 
sources (CLASP), Coalition on Adult Basic Education, Community College Research Center, Council for 
recommended by Adult and Experiential Learning, Council for Advancement of Adult Literacy, City University of 
experts in the New York (CUNY)/LaGuardia Community College, Dollar General Literacy Foundation, 

field Education Testing Services, Gates Foundation, GED Testing Service, Goodling Institute, 


Harvard University Graduate School Education, Institute for Study of Adult Literacy, Jobs for 
the Future, Literacy Education and Second Language Learning for Adults, Literacy 
Information Network Communication System (LINCS), Lumina Foundation, Migration Policy 
Institute, National Association for Adults with Special Learning Needs (NAASLN), National 
Center for the Study of Adult Learning and Literacy (NCSALL), National Coalition for Literacy 
(NCL), National College Access Network, National College Transition Network (NCTN), National 
Council of State Directors of Adult Education (NCSDAE), National Skills Coalition, U.S. 
Department of Education, Office of Career & Technical Education (OCTAE) Resources, 
Portland State University/Longitudinal Study of Adult Learning, RTI International, University 
of Kansas Center for Research on Learning Division of Adult Studies, University of Tennessee, 
Center for Literacy, Education, & Employment (CLEE), VALUEUSA, Washington State Board 
for Community and Technical Colleges, World Education 


A-15 


Request that researchers, practitioners, and policymakers also submit relevant 
studies. 


Researchers, practitioners, and policymakers in adult education were asked to identify research studies 
they thought might be relevant. This step was taken to help identify research that might not show up as 
readily in searches of electronic databases. 


A.4. Screening and Reviewing Studies 


References identified through the searches were first screened to determine whether they were indeed 
about adult education. If the references were relevant, they were then further screened to determine 
whether they met the other eligibility requirements described previously. The eligible references were 
then reviewed to determine whether they conformed to the WWC’s standards of rigor. 


Screen references using the What Works Clearinghouse’s systematic procedures to 
identify eligible studies. 


References identified through the broad literature search were screened for eligibility by WWC- 
certified staff. 


Short abstracts were obtained for all references identified in the literature search. These abstracts were 
then screened to determine which ones fell within the scope of the review effort and were therefore 
eligible for review. Abstracts that clearly were not eligible (for example, not relevant to adult 
education) were screened out. When additional information was required to make an eligibility 
determination, the screeners read through the full text of the reference to determine eligibility. When 
they were unsure of eligibility, they flagged a study for a more senior staff person to review. 


Although many references were identified through the broad literature search, few examined the 
impacts of strategies in adult education. 


The broad literature search yielded 4,199 references, which included journal articles, reports, 
conference proceedings, dissertations, and theses. Almost half (45 percent) of the references were 
screened out because they were not relevant to adult education. Of those still relevant, others were 
screened out because they were not research studies but rather newspaper articles or blog posts (38 
percent), did not examine the effectiveness of an eligible adult education strategy (34 percent), and/or 
did not use an eligible research design (15 percent). Another 10 percent were screened out for some 
other reason (for example, sample member characteristics, outcomes, location, or a publication date 
did not fit review criteria). Only about three percent of the relevant studies examined the impacts of 
strategies in adult education on improving the academic skills, educational progress, employment or 
earnings of learners. 


Some of the eligible references reported on the same evaluation, whereas others reported on 
multiple evaluations. 


In some cases, multiple references reported on findings from the same evaluation. For example, one 
reference may have reported on one-year impacts and another on two-year impacts. Because these 
references were reporting on findings from the same evaluation, they were considered together as a 
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single study (following WWC procedures). In other cases, a reference may have reported on multiple 
evaluations. This occurred, for example, if study authors examined more than two groups of 
individuals and each group received a different strategy, or in some cases, no strategy. In an example 
of three groups: Strategy A, Strategy B, and a group that received no strategy, each comparison that the 
authors made between these three groups - Strategy A versus no strategy, Strategy B versus no 
strategy, Strategy A versus Strategy B - was considered a separate study. After applying the WWC’s 
definition of a study, 54 studies were determined to be eligible and assigned for review. 


Review studies using the What Works Clearinghouse’s systematic procedures to 
identify rigorous evidence. 


Studies were reviewed by WWC-certified staff and recorded in the WWC’s online study review 
guide. 


The review used the Review of Individual Studies Protocol (RISP) version 4.0 because the WWC did not 
have a review protocol specific to adult 
education. The RISP is a broad, flexible 
review protocol that includes a wide range of 
outcomes, many relevant to adult education. 


The WWC applies one of three study ratings to each 
eligible study: 


Meets WWC standards without reservations: The 
WWC has a high degree of confidence in at least one of the 
study’s findings regarding the effects of the strategy being 
tested. 


: Meets WWC standards with reservations: The WWC 
WWC Standards Handbook version 4.0. : : 
has a medium degree of confidence in at least one of the 


ewe recorded their review, including study’s findings regarding the effects of the strategy being 
details of the study and methods used, such tached. 


Each of the 54 studies was assigned to at 
least one certified reviewer who reviewed 
the study using the RISP version 4.0 and the 


as the Way a wich sample members WEES Does not Meet WWC standards: The WWC does not 
assigned to receive the strategy being tested have confidence in the study’s findings regarding the 

or not, any attrition or data missing for effects of the strategy being tested. 

sample members, and whether groups aise Source: Adapted from the What Works Clearinghouse Glossary. 
similar at the start of the study in the WWC’s https://ies.ed.gov/ncee/wwe/Glossary 


online study review guide (OSRG). Each 
eligible analysis, or comparison in a study was reviewed against WWC standards and received one of 
three WWC study ratings: Meets WWC standards without reservations, Meets WWC standards with 
reservations, and Does not meet WWC standards. 


A study was independently reviewed by a second certified reviewer if the first reviewer indicated that 
(1) the study did not meet standards but a senior researcher, also known as a reconciler disagreed or (2) 
if the first reviewer indicated that the study met WWC standards, or could meet standards with more 
data provided by the study author. 


When some information needed to determine a study rating was missing, authors were contacted with 
a request to provide the needed information. These requests did not ask authors to perform new or 
additional analyses. All information received through an author query was documented in the OSRG. 
Reconcilers, were responsible for helping to draft any author queries, ensuring that the standards were 
applied correctly throughout the review, confirming the study’s rating, resolving any discrepancies 
between reviewers, and creating a final record in the OSRG. 


Twenty-two studies met WWC design standards with or without reservations. In all 22, study authors 
compared the outcomes of participants who were in a group offered an adult education strategy 
(referred to by WWC as the intervention or treatment group) to the outcomes of a group that was not 
offered that same adult education strategy (referred to by WWC as the comparison group). In some 
cases, the comparison group may have received a set of services different from those provided to the 
treatment group; in others, a comparison group may not have received any services. Eleven studies 
received the WWC’s highest rating of “meets standards without reservations”— because they were well- 
executed randomized controlled trials. The other 11 studies met standards with reservations.” Many of 
these studies were quasi-experimental designs, in which the research groups were determined to be 
similar before receiving the strategy. 


When available, information was recorded for subgroups. 


As part of the review process, reviewers recorded information about subgroups, when available. 
Subgroups may be important to consider because, although a strategy may not work for everyone, it 
may work for some groups of individuals. A major emphasis of WIOA is to examine whether strategies 
are effective for learners with particular barriers to employment. Thus, reviewers recorded 
information, if available, for subgroups defined by the following: 


e Barriers to employment—displaced homemakers; eligible migrant and seasonal farmworkers; ex- 
offenders; homeless individuals; Native Americans, Alaska Natives, and Native Hawaiians; 
individuals with disabilities, including those in receipt of Social Security Disability Insurance; 
individuals within two years of exhausting lifetime Temporary Assistance for Needy Families 
(TANF) eligibility; long-term unemployed individuals (unemployed for 27 or more consecutive 
weeks); low-income individuals; older individuals (55 years and older); single parents (including 
single pregnant women); disconnected youth (youth between ages 16 and 24 who are out of school 
and work); English language learners 


e Demographics—gender, age, race/ethnicity 


e Baseline skill level—literacy, numeracy, English language skills 


All eligible findings were designated as main or supplemental findings following WWC procedures. 
Only main findings are reported in the evidence review snapshot and Appendix B. 


The WWC Procedures Handbook version 4.0 provides guidance to reviewers when determining 
whether findings should be considered as main or supplemental. This determination depends first on 
which analyses meet WWC standards. Generally, main findings are based on the full sample of 
participants and use the most aggregate measure of the outcome measure, for example a full-scale 
score instead of individual subscale scores. Further, the RISP version 4.0 considers findings based on 
data collected closest to the end of the exposure to the strategy as main, and for some types of 
outcomes, such as educational attainment and labor market, the RISP considers both the shortest and 


? In cases where different analyses received different ratings, the overall WWC study rating was the highest rating across them. 
This approach reflects the credibility of evidence from the study, per the version 4.0 Procedures Handbook. 
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longest follow-up findings as main in separate short-run and long-run outcome domains. Other findings 
are considered supplemental. 


In addition to documenting the main findings, reviewers also entered data into the OSRG on all other 
eligible supplemental outcomes reported by authors. The most common supplemental outcomes were 
additional time points not prioritized as main findings per the RISP. All findings, both main and 
supplemental, can be found by searching the WWC’s Reviews of Individual Studies, available through 
the WWC’s website. 


A.5. Summarizing and Characterizing Findings 


Summarize RISP outcome domains within four key areas of adult education learner 
outcomes: academic skills, educational progress, employment, and earnings. 


The RISP protocol, which was designed to be flexible to accommodate studies in a wide range of 
education areas, includes many outcome groups, referred to as domains. Many of these domains are 
relevant to adult education. Table A.4 lists the 16 outcome domains that contained at least one main 
finding that met standards across all reviewed studies. In the evidence review snapshot, these 16 
outcome domains were summarized within four key areas of adult education learner outcomes— 
academic skills, educational progress, employment, and earnings. Table A.4 maps the RISP outcome 
domains to these four areas of outcomes. The WWC’s Reviews of Individual Studies, available through 
the WWC’s website, contains the full review of each of the reviewed studies, including the full array of 
RISP domains. 


Table A.4. Mapping of RISP outcome domains to adult education outcome areas: 
academic skills, educational progress, employment, and earnings 
Adult education outcome 
RISP version 4.0 domains areas 
Alphabetics, comprehension, reading fluency, general mathematics, literacy Academic skills 
achievement, and English language proficiency 
Progressing in college, industry-recognized credential, certificate, or license (short- Educational progress 
run), college enrollment, completing high school (short-run), college degree 
attainment (short- and long-run) 
Employment (short- and long-run) Employment 
Earnings (short- and long-run) Earnings 


Characterize study findings on the four key areas of learner outcomes using What 
Works Clearinghouse procedures. 


To simplify the summary of study findings, findings on academic skills, educational progress, 
employment, and earnings were characterized as a (i) statistically significant positive effect, (ii) 
substantively important positive effect, (iii) indeterminate effect, (iv) substantively important negative 
effect, or (v) statistically significant negative effect. These classifications follow the WWC’s 
characterization of findings within a domain for individual studies as laid out in the WWC Procedures 
Handbook version 4.0. Tables IV.1 and IV.2 in the Procedures Handbook were used to characterize the 
findings of an effect based on the direction (positive, negative, indeterminate), and strength of outcome 
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findings as measured by the estimated effect size in standard deviation units. Under version 4.0, the 
WWC considered effect sizes of 0.25 standard deviations or larger as substantively important. 


The characterization also accounted for any necessary multiple comparisons corrections, following 
WWC procedures. If authors examined multiple outcomes within a RISP outcome domain, for 
example, multiple tests of reading fluency within the reading fluency outcome domain, the Benjamini- 
Hochberg (BH) correction was used to account for multiple comparisons. This correction guards 
against observing spurious impact findings, and thus mistakenly concluding that there were differences 
between groups when authors conducted multiple tests of outcomes within the same domain. When 
summarizing the RISP outcome domains within the four key areas of adult learner outcomes, no 
additional corrections for multiple comparisons were made. 


A.6. Reporting Findings 


Simplify the characterization of findings to positive, negative, no effect, or mixed 
effects. 


For reporting purposes, the five possible WWC characterizations of findings were simplified to positive 
effects, negative effects, no effects, and mixed effects (Table A.5). Only statistically significant effects 
were categorized as positive or negative. For example, the WWC characterization of a ‘Statistically 
significant positive effect” was simplified to a “Positive effect” for this review. Substantively important 
effects, that is impacts with an effect size of .25 or greater but that were not statistically significant, 
were treated as no effects because large effect sizes from studies with small samples are not necessarily 
informative about the effect that may be observed in a study that uses a larger sample; this is consistent 
with the direction the WWC has moved in treating effects that are not significant .* The WWC’s 
‘Indeterminate effect’ characterization was split into ‘No effect’ if the finding(s) within a domain or 
outcome area were not statistically significant or ‘Mixed effects’ if the findings included both 
statistically significant positive and negative effects. 


Table A.5. Final characterization of findings 
What Works Clearinghouse characterization of findings from the Adult education evidence review 
version 4.0 Procedures Handbook characterization of findings 
Statistically significant positive effect Positive effect 
Substantively important positive effect No effect 
Indeterminate effect No effect@ 
Mixed effects” 
Substantively important negative effect No effect 
Statistically significant negative effect Negative effect 


4 “No effect” indicates that the finding(s) within a domain or outcome area were not statistically significant. 


> «Mixed effect” indicates the findings within a domain or outcome area include both statistically significant positive and 
statistically significant negative effects. 


3 Under the WWC’s 4.1 Standards and Procedures Handbooks that were released after this review was conducted, the 
substantively important characterizations were removed. Thus, effect sizes are only characterized by their sign and statistical 
significance, consistent with the approach taken for this evidence review. 
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The evidence review snapshot reports the characterizations of findings at the level of the adult 
education outcome areas, while Appendix B includes characterizations of findings for the RISP 
outcome domains. For example, consider a study that measured alphabetics, comprehension, and 
reading fluency, which all fall into the academic skills outcome area. The snapshot would report a 
single characterization for the academic skills outcome area, while Appendix B would include the 
characterizations for each of these RISP outcomes. 


Organize findings using adult education categories and sample strategies. 


The report organizes findings using a field-informed structure to identify categories of adult education 
strategies for which rigorous evidence does and does not exist (Exhibit A.1). In addition, this structure 
was used to identify whether findings were consistent within categories and strategies. 
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Appendix B. 


Details of Reviewed Studies 


B-1 


This appendix contains information about the studies that were reviewed, which are those studies that 
made it through the screening process. References are included for each of the 54 studies that were 
reviewed, along with the WWC rating each study received and the reason for the rating. For the 22 
studies that met WWC standards, summaries about the strategies tested and technical details of the 
study design are also provided. While some technical terms are defined in Exhibit B.1 this appendix 
was written for an audience with familiarity in research design. 


Exhibit B.1. Terms and definitions used in the details of reviewed studies 


e WWC study rating: One of three ratings that the WWC applies to a study. Each study was assigned the 
highest rating earned across all eligible analyses. The rating reflects the credibility of evidence from the study, 
per the version 4.0 Procedures Handbook. The three ratings are: 

— Meets WWC standards without reservations: The WWC has a high degree of confidence in at least one of 
the study’s findings regarding the effects of the strategy being tested 
Meets WWC standards with reservations: The WWC has a medium degree of confidence in at least one of 
the study’s findings regarding the effects of the strategy being tested 
Does not meet WWC standards: The WWC does not have confidence in the study’s findings regarding the 
effects of the strategy being tested. 


Setting: Where the study took place, including the geographic location and specific environment like a 
classroom. 


Methods: The procedures researchers used to examine whether a strategy was effective. This often includes 
details of the research design including the way in which participants were assigned to receive the strategy 
being tested or not. 


Eligible research designs: Methods from which conclusions about effectiveness can be drawn and for which 
WWC standards and procedures can be applied. 

— Randomized controlled trial (RCT): When individuals or clusters of individuals in a study are randomly 
assigned to either receive the strategy being tested or not receive the strategy through a process like a 
lottery or coin toss. The groups are then compared to determine if the strategy was effective at improving 
outcomes like skills, educational progress, employment, and earnings. 

Quasi-experimental design (QED): When decisions, about which individuals or clusters of individuals in a 
study receive the strategy being tested, are made using a process that is not random. The groups are then 
compared to determine if the strategy was effective at improving outcomes. 

Regression Discontinuity Design (RDD): When individuals are assigned to receive a strategy or not based 
on a specific threshold on a continuous scale (such as scoring above a cut-off number on an achievement 
test). The groups are then compared to determine if the strategy was effective at improving outcomes. 
Single-Case Design (SCD): When individuals receive the strategy being studied some, but not all the time. 
Outcomes for the individual are measured at multiple times, both when they are receiving the strategy 
and not, to determine if the presence of the strategy improved their outcomes. 


Study sample: Participants in a study, including the learners and/or classrooms that received and did not 
receive the strategy being tested. 

Treatment group: Group offered the adult education strategy that was being tested. The WWC also refers to 
this as the intervention group. 


Comparison group: Group that was not offered the adult education strategy being tested. 


Outcomes: Results that are measured to determine if the strategy was effective. 


Implementation: How a strategy is carried out, including information about how it was provided to sample 
members. 


Source: Adapted from the What Works Clearinghouse Glossary. https://ies.ed.gov/ncee/wwe/Glossary 
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B.1. References for all Studies Reviewed 


This section lists the references for all 54 studies that meet review eligibility requirements following the 
What Works Clearinghouse (WWC) Standards and Procedures Handbooks and Review of Individual 
Studies Protocol version 4.0. Key requirements are that the study examined the effectiveness of an 
eligible adult education strategy using an eligible research design, that is, a design that provides 
evidence that observed impacts are due to the strategy being tested. All the requirements are described 
in Appendix A. 


The references are organized by their WWC rating, which is based on the highest rating assigned to any 
analysis reported in the study. The WWC rating indicates the degree of confidence that any effect 
observed was due to the strategy tested. This section also includes the reason for the rating, which is 
based on technical aspects of the research design. There is not always a one-to-one alignment between 
a study and a reference (see Appendix A). For instance, a single study could be described in multiple 
references, in which case additional references are listed as an additional source. Also, a single 
reference could report on multiple studies, in which case a separate rating was determined and 
reported for each study. 


Meets WWC standards without reservations 


Batchelder, J. S., & Rachal, J. R. (2000a). Efficacy of a computer-assisted instruction program in a 
prison setting: An experimental study. Adult Education Quarterly, 50(2), 120-133. (Meets WWC 
standards without reservations. The study is a randomized controlled trial with low attrition.) 


Additional source: 
Batchelder, J. S., & Rachal, J. R. (2OOOb). Effects of a computer-assisted instruction program in a 
prison setting: An experimental study. Journal of Correctional Education, 51(4), 324-332. 

Condelli, L., Cronen, S., Bos, J., Tseng, F., & Altuna, J. (2010). The impact of a reading intervention for 
low-literate adult ESL learners (NCEE 2011-4003). U.S. Department of Education, Institute of 
Education Sciences, National Center for Education Evaluation and Regional Assistance. 
https://ies.ed.gov/ncee/pubs/20114003/pdf/20114003.pdf (Meets WWC standards without 
reservations. The study is a cluster randomized controlled trial with low attrition at the cluster and 
individual levels.) 


Cook, R., Hamadyk, J., Zeidenberg, M., Rolston, H., & Gardiner, K. (2018). Madison Area Technical 
College Patient Care Pathway program: Implementation and early impact report (OPRE Report No. 
2018-48). U.S. Department of Health and Human Services, Administration for Children and 
Families, Office of Planning, Research, and Evaluation. (Meets WWC standards without 
reservations. The study is a randomized controlled trial with low attrition.) 


Additional source: 

Cook, R., Hamadyk, J., Zeidenberg, M., Rolston, H., & Gardiner, K. (2018). Madison Area Technical 
College Patient Care Pathway program: Implementation and early impact report (appendices) 
(OPRE Report No. 2018-48). U.S. Department of Health and Human Services, Administration for 
Children and Families, Office of Planning, Research, and Evaluation. 
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Glosser, A., Martinson, K., Cho, S. W., & Gardiner, K. (2018). Washington State’s Integrated Basic 
Education and Skills Training (I-BEST) program in three colleges: Implementation and early impact 
report (OPRE Report No. 2018-87). U.S. Department of Health and Human Services, Administration 
for Children and Families, Office of Planning, Research, and Evaluation. (Meets WWC standards 
without reservations. The study is a randomized controlled trial with low attrition.) 


Additional source: 


Glosser, A., Martinson, K., Cho, S. W., & Gardiner, K. (2018). Washington State’s Integrated Basic 
Education and Skills Training (I-BEST) program in three colleges: Implementation and early impact 
report (appendices) (OPRE Report No. 2018-87). U.S. Department of Health and Human Services, 
Administration for Children and Families, Office of Planning, Research, and Evaluation. 

Gray, S. H., Ehri, L. C., & Locke, J. L. (2018). Morpho-phonemic analysis boosts word reading for adult 
struggling readers. Reading & Writing, 3101), 75-98. (Meets WWC standards without reservations. 
The study is a randomized controlled trial with low attrition.) 


Hamadyk, J., & Zeidenberg, M. (2018). Des Moines Area Community College Workforce Training Academy 
Connect Program: Implementation and early impact report (OPRE Report No. 2018-82). U.S. 
Department of Health and Human Services, Administration for Children and Families, Office of 
Planning, Research, and Evaluation. (Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition.) 


Additional source: 


Hamadyk, J., & Zeidenberg, M. (2018). Des Moines Area Community College Workforce Training 
Academy Connect Program: Implementation and early impact report (appendices) (OPRE Report 
No. 2018-82). Washington, DC: U.S. Department of Health and Human Services, Administration for 
Children and Families, Office of Planning, Research, and Evaluation. 

Hock, M. F., & Mellard, D. F. (2011). Efficacy of learning strategies instruction in adult education. 
Journal of Research on Educational Effectiveness, 4(2), 134-153. (This reference included four 
separate studies and each study received its own review and rating. Only one of these studies 
received the Meets WWC standards without reservations rating.) 


Study 1: Summarization Strategy. (Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition.) 

Martin, V., & Broadus, J. (2013). Enhancing GED instruction to prepare students for college and careers: 
Early success in LaGuardia Community College’s Bridge to Health and Business Program. MDRC. 
(Meets WWC standards without reservations. The study is a randomized controlled trial with low 
attrition.) 


Miller, C., Millenky, M., Schwartz, L., Goble, L., & Stein, J. (2016). Building a future: Interim impact 
findings from the YouthBuild evaluation. MDRC. (Meets WWC standards without reservations. The 
study is a randomized controlled trial with low attrition.) 

Additional sources: 

Miller, C., Cummings, D., Millenky, M., Wiegand, A., & Long, D. (2018). Laying a foundation: Four- 
year results from the national YouthBuild evaluation. MDRC. 

Wiegand, A., Manno, M., Leshnick, S., Treskon, L., Geckeler, C., Lewis-Charp, H, ... Nicholson, B. 
(2015). Adapting to local context: Findings from the YouthBuild evaluation implementation study. 
MDRC. 
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Modicamore, D., Lamb, Y., Taylor, J., Takyi-Laryea, A., Karageorge, K., & Ferroggiaro, E. (2017). 
Accelerating connections to employment. Vol. I. Final evaluation report. ICF. (Meets WWC standards 
without reservations. The study is a randomized controlled trial with low attrition.) 


Additional source: 
Modicamore, D., Lamb, Y., Taylor, J., Takyi-Laryea, A., Karageorge, K., & Ferroggiaro, E. (2017). 
Accelerating connections to employment. Vol. IT. Final evaluation report appendices. ICF. 

Robinson, S. A. (2018). A study designed to increase the literacy skills of incarcerated adults. Journal of 
Correctional Education, 69(1), 60-72. (Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition.) 


Meets WWC standards with reservations 


Alamprese, J., MacArthur, C., Price, C., & Knight, D. (2011). Effects of a structured decoding curriculum 
on adult literacy learners’ reading development. Journal of Research on Educational Effectiveness, 
A(2), 154-172. (This reference included two separate studies and each study received its own review 
and rating.) 


Study 1: Making Sense of Decoding and Spelling (Meets WWC standards with reservations. The 
study is a cluster randomized controlled trial with high individual-level attrition, and the groups of 
learners were shown to be similar at the start of the study.) 

Study 2: K-3 curriculum adapted for adult learners (Meets WWC standards with reservations. The 
study used a cluster quasi-experimental design, and the groups of learners were shown to be 
similar at the start of the study.) 

Anderson, T., Kuehn, D., Eyster, L., Barnow, B., & Lerman, R. (2017). New evidence on integrated career 
pathways: Final impact report for Accelerating Opportunity. Urban Institute. (Meets WWC standards 
with reservations. The study used a quasi-experimental design, and the groups of learners were 
shown to be similar at the start of the study.) 


Additional sources: 

Anderson, T., Eyster, L., Lerman, R., Clymer, C., Conway, M., & Montes, M. (2014). The first year of 
Accelerating Opportunity: Implementation findings from the states and colleges. Urban Institute. 
http://www.urban.org/research/publication/first-year-accelerating-opportunity-implementation- 
findings-states-and-colleges 

Anderson, T., Eyster L., Lerman, R., O’Brien, C., Conway, M., Jain, R., & Montes, M. (2015). The 
second year of Accelerating Opportunity: Implementation findings from the states and colleges. Urban 
Institute. http://www.urban.org/research/publication/second-year-accelerating-opportunity- 
implementation-findings-states-and-colleges 

Anderson, T., Eyster L., Lerman, R., O’Brien, C., Conway, M.., Jain, R., & Montes, M. (2016). 
Implementation of Accelerating Opportunity: Lessons from the field. Urban Institute. 
http://www.urban.org/research/publication/implementation-accelerating-opportunity-final- 
implementation-findings-lessons-field 


Eyster, L., Anderson, T., Lerman, R., Kuehn D., Barnow, B., Conway, M., ... Montes, M. (2018). 
Findings from the Accelerating Opportunity evaluation: Building the evidence on integrated career 
pathways. Urban Institute. 
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Greenberg, D., Wise, J., Morris, R., Fredrick, L., Nanda, A., & Hye-K., P. (2011). A randomized control 
study of instructional approaches for struggling adult readers. Journal of Research on Educational 
Effectiveness, 4(2), 101-117. doi:10.1080/19345747.2011.555288 (This reference included four separate 
studies and each study received its own review and rating.) 


Study 1: Decoding and Fluency (Meets WWC standards with reservations. The study is a 
compromised randomized controlled trial, and the groups of learners were shown to be similar at 
the start of the study.) 


Study 2: Decoding, Comprehension, and Fluency (Meets WWC standards with reservations. The 
study is a compromised randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study.) 

Study 3: Decoding, Comprehension, Fluency, and Extensive Reading (Meets WWC standards with 
reservations. The study is a compromised randomized controlled trial, and the groups of learners 
were shown to be similar at the start of the study.) 

Study 4: Extensive Reading (Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be similar at the start of the 
study.) 

Additional source: 

Rodrigo, V., Greenberg, D., & Segal, D. (2014). Changes in reading habits by low literate adults 
through extensive reading. Reading in a Foreign Language, 26(1), 73-91. 

Hock, M. F., & Mellard, D. F. (2011). Efficacy of learning strategies instruction in adult education. 
Journal of Research on Educational Effectiveness, 4(2), 134-153. (This reference included four 
separate studies and each study received its own review and rating. Only one of these studies 
received the Meets WWC standards with reservations rating.) 


Study 2: Prediction Strategy (Meets WWC standards with reservations. The study is a randomized 
controlled trial with high attrition, and the groups of learners were shown to be similar at the start 
of the study.) 

Sabatini, J. P., Shore, J., Holtzman, S., & Scarborough, H. S. (2011). Relative effectiveness of reading 
intervention programs for adults with low literacy. Journal of Research on Educational Effectiveness, 
A(2), 118-133. (This reference included three separate studies and each study received its own 
review and rating.) 


Study 1: Corrective Reading (Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be similar at the start of the 
study.) 

Study 2: Guided Reading (Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be similar at the start of the 
study. 

Study 3: RAVE-O (Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be similar at the start of the 
study.) 

Additional source: 

Scarborough, H. S., Sabatini, J. P., Shore, J., Cutting, L. E., Pugh, K., & Katz, L. (2013). Meaningful 
reading gains by adult literacy learners. Reading and Writing: An Interdisciplinary Journal, 26(A4), 
593-613. 


B-7 


Does not meet WWC standards 


Alewine, H. S. (2010). Andragogical methods and readiness for the correctional GED classroom. Journal 
of Correctional Education, 61(1), 9-22. (Does not meet WWC standards. The measures of 
effectiveness cannot be attributed solely to the intervention.) 


Autrey, J. H. (1999). Effects of direct instruction and precision teaching on achievement and persistence 
of adult learners (Doctoral dissertation). http://digitalcommons.wayne.edu/oa dissertations/1763 
(Does not meet WWC standards. The measures of effectiveness cannot be attributed solely to the 
intervention.) 


Berry, A. B., & Mason, L. H. (2012). The effects of self-regulated strategy development on the writing of 
expository essays for adults with written expression difficulties: Preparing for the GED. Remedial 
and Special Education, 33(2), 124-136. (Does not meet WWC pilot single-case design standards. The 
study had insufficient data to evaluate the attempts to demonstrate an intervention effect.) 


Bingham, M. J. (2002). Effects of computer-assisted instruction versus traditional instruction on adult GED 
student TABE scores (Doctoral dissertation). ProQuest Dissertations and Theses database. (UMI No. 
3049561) (Does not meet WWC standards. The study used a quasi-experimental design and the 
groups of learners are not shown to be similar at the start of the study.) 


Blackmer, R., & Hayes-Harb, R. (2016). Identifying effective methods of instruction for adult emergent 
readers through community-based research. Journal of Research and Practice for Adult Literacy, 
Secondary, and Basic Education, 5(2), 35-49. (Does not meet WWC standards. The study used a 
quasi-experimental design and the groups of learners are not shown to be similar at the start of the 
study.) 


Frazier Varner, D. (2010). Effect of instructional design on academic success of adult basic education 
learners: Individualized versus group design (Doctoral dissertation). ProQuest Dissertations and 
Theses database. (UMI No. 3398862) (Does not meet WWC standards. The study used a quasi- 
experimental design and the groups of learners are not shown to be similar at the start of the 
study.) 


Grafals, Z. (2013). English learning predictors of listening and speaking self-efficacy for adult second 
language learners (Doctoral dissertation). ProQuest Dissertations and Theses database. (UMI No. 
3572507) (Does not meets WWC standards. The study used a quasi-experimental cluster design and 
the clusters are not shown to be similar at the start of the study.) 


Hock, M. F., & Mellard, D. F. (2011). Efficacy of learning strategies instruction in adult education. 
Journal of Research on Educational Effectiveness, 4(2), 134-153. (This reference included four 
separate studies and each study received its own review and rating. Only two of these studies 
received the Does not meet WWC standards rating.) 


Study 3: Bridging Strategy (Does not meet WWC standards. The study is a randomized controlled 
trial with high attrition and the groups of learners are not shown to be similar at the start of the 
study.) 

Study 4: Fluency Strategy (Does not meet WWC standards. The study is a randomized controlled 
trial with high attrition and the groups of learners are not shown to be similar at the start of the 
study. 
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Huang, J., & Newbern, C. (2012). The effects of metacognitive reading strategy instruction on reading 
performance of adult ESL learners with limited English and literacy skills. Journal of Research and 
Practice for Adult Literacy, Secondary, and Basic Education, 1(2), 66-77. (Does not meet WWC 
standards. The measures of effectiveness cannot be attributed solely to the intervention.) 


Jenkins, D., Zeidenberg, M., & Kienzl, G. (2009). Educational outcomes of I-BEST, Washington State 
Community and Technical College System’s Integrated Basic Education and Skills Training program: 
Findings from a multivariate analysis. (CCRC Working Paper No. 16). Columbia University, Teachers 
College, Community College Research Center. (This reference included two separate studies and 
each study received its own review and rating.) 


Study 1: Difference-in-difference analyses (Does not meets WWC standards. The study used a 
quasi-experimental cluster design and the clusters are not shown to be similar at the start of the 
study.) 

Study 2: Ordinary least squares or unadjusted means (Does not meet WWC standards. The study 
used a quasi-experimental design and the groups of learners are not shown to be similar at the start 
of the study.) 

Additional source: 

Zeidenberg, M., Cho, S., & Jenkins, D. (2010). Washington State’s Integrated Basic Education and 
Skills Training program (I-BEST): New evidence of effectiveness (CCRC Working Paper No. 20). 
Columbia University, Teachers College, Community College Research Center. 

Jolly, K. F. M. (2000). The effects of the use of the writing process on the reading comprehension skills 
of rural adult basic education students (Doctoral dissertation). ProQuest Dissertations and Theses 
database. (UMI No. 9984536) (Does not meet WWC standards. The measures of effectiveness 
cannot be attributed solely to the intervention.) 


Kim, Y. (2008). The role of task-induced involvement and learner proficiency in L2 vocabulary 
acquisition. Language Learning, 58(2), 285-325. (This reference included four separate studies and 
each study received its own review and rating.) 


Study 1: Composition (Does not meets WWC standards. The study is a compromised randomized 
controlled trial and the groups of learners are not shown to be similar at the start of the study.) 
Study 2: Gap-fill (Does not meets WWC standards. The study is a compromised randomized 
controlled trial and the groups of learners are not shown to be similar at the start of the study.) 
Study 3: Reading (Does not meets WWC standards. The study is a compromised randomized 
controlled trial and the groups of learners are not shown to be similar at the start of the study.) 
Study 4: Sentence Writing (Does not meets WWC standards. The study is a compromised 
randomized controlled trial and the groups of learners are not shown to be similar at the start of 
the study.) 

Klinghoffer, C. L. (2008). Situational dialogues in a community college: English as a second language 
curriculum (Doctoral dissertation). https://files.eric.ed.gov/fulltext/ED504218.pdf (Does not meet 
WWC standards. The measures of effectiveness cannot be attributed solely to the intervention.) 


Krause, T. (2018). Using VocabularySpellingCity with adult ESOL students in community college. 
ORTESOL Journal, 35, 43-46. (Does not meet WWC standards. The study used a quasi-experimental 
design and the groups of learners are not shown to be similar at the start of the study.) 
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Ksoll, C., Aker, J., Miller, D., Perez, K., & Smalley, S. (2014). Learning without teachers? A randomized 
experiment of a mobile phone-based adult education program in Los Angeles (CGD Working Paper 368). 
Center for Global Development. https://www.cgdev.org/publication/learning-without-teachers- 
randomized-experiment-mobile-phone-based-adult-education (Does not meet WWC standards. The 
study includes only outcomes for which the measures were collected differently for subjects in the 
intervention and comparison groups.) 


Lynch, E. M. (2016). Faith is confidence: The implication of psychosocial components in faith-based 
educational programs on expressive communication skills of adult learners. Journal of Research on 
Christian Education, 25(2), 169-187. (Does not meets WWC standards. The study used a quasi- 
experimental cluster design and the clusters are not shown to be similar at the start of the study.) 


Maisak, N. (2017). Transitioning students from adult education to postsecondary education through co- 
enrollment career pathways model (Doctoral dissertation). ProQuest Dissertations and Theses 
database. (UMI No. 10619502) (Does not meet WWC standards. The study used a quasi- 
experimental design and the groups of learners are not shown to be similar at the start of the 
study.) 


Massengill, D. (2003). Guided reading—An instructional framework for adults. Adult Basic Education, 
13(3), 168-188. (Does not meet WWC pilot single-case design standards. The study had insufficient 
data to evaluate the attempts to demonstrate an intervention effect.) 


Additional source: 
Massengill, D. (2004). The impact of using guided reading to teach low-literate adults. Journal of 
Adolescent & Adult Literacy, 47(7), 588-602. 

Patterson, M. B. (2016). “A big and excellent opportunity for my future”: Adult Learner Leadership in 
Education Services (ALLIES) evaluation year 1 survey, assessment, observation, and program 
information findings. VALUEUSA. (Does not meets WWC standards. The study is a compromised 
cluster randomized controlled trial and the clusters are not shown to be similar at the start of the 
study.) 

Additional sources: 

Patterson, M. B. (2016). ALLIES final year leadership report: Part 1, Key quantitative findings. 
VALUEUSA. 

Patterson, M. B. (2017). Adult Learner Leadership in Education Services: What VALUEUSA learned 
about learners as leaders. Journal of Research and Practice for Adult Literacy, Secondary, and Basic 
Education, 6(3), 35-49. 

Paulson, U., & Patterson, M. B. (2017). ALLIES final year leadership report: Part 3, Connections of 
critical thinking measures with learner experiences. VALUEUSA. 

Prins, E., Drayton, B., Gungor, R., & Kassab, C. (2010). GED preparation through distance learning in 
rural Pennsylvania. The Pennsylvania State University. (This reference included three separate 
studies and each study received its own review and rating.) 


Study 1: Distance Learning - Blended (Does not meet WWC standards. The study used a quasi- 
experimental design and the groups of learners are not shown to be similar at the start of the 
study.) 

Study 2: Distance Learning - Pure (Does not meet WWC standards. The study used a quasi- 
experimental design and the groups of learners are not shown to be similar at the start of the 
study.) 
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Study 3: Distance Learning ((Does not meet WWC standards. The study used a quasi-experimental 
design and the groups of learners are not shown to be similar at the start of the study.) 

Additional sources: 

Prins, E., Drayton, B., Gungor, R., & Kassab, C. (2011). GED preparation through distance learning in 
rural Pennsylvania. Center for Rural Pennsylvania. 

Prins, E., Drayton, B., Gungor, R., & Kassab, C. (2012). Distance learning for GED students in rural 
Pennsylvania. American Journal of Distance Education, 26(4), 217-235. 

Reed, T. M. (2014). Inmate educational program and recidivism rates of violent and nonviolent jail inmates 
(Doctoral dissertation). ProQuest Dissertations and Theses database. (UMI No. 3646207) (Does not 
meet WWC standards. The study used a quasi-experimental design and the groups of learners are 
not shown to be similar at the start of the study.) 


RISE Partnership. (2014). Wisconsin’s Regional Industry Skills Education (RISE) Career Pathway Bridges: 
An evaluation of Career Pathway Bridge programming in Wisconsin 2012-2014. Wisconsin Career 
Pathways. (Does not meet WWC standards. The study used a quasi-experimental design and the 
groups of learners are not shown to be similar at the start of the study.) 


Shaw, D. M., & Berg, M. A. (2008). Effects of a word study intervention on spelling accuracy among 
low-literate adults. Adult Basic Education and Literacy Journal, 2(3), 131-139. (Does not meet WWC 
standards. The study used a quasi-experimental design and the groups of learners are not shown to 
be similar at the start of the study.) 


Warren, R. D. (2012). The effect of adaptive confidence strategies in computer-assisted instruction on 
learning and learner confidence (Doctoral dissertation). 
https://diginole.lib.fsu.edu/islandora/object/fsu:183178/datastream/PDF/view (This reference 
included two separate studies and each study received its own review and rating.) 


Study 1: Program Control (Does not meet WWC standards. The eligible outcomes do not meet 
WWC requirements.) 

Study 2: Shared Control (Does not meet WWC standards. The eligible outcomes do not meet WWC 
requirements.) 
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B.2. Study Details for the 22 Studies that Met What Works 
Clearinghouse Standards 


Within a study that meets WWC standards, each analysis in the study is judged against WWC standards. 
Thus, there may be analyses in a study that receive different ratings. Details of the 22 studies that met 
WWC standards are provided in this section. This information is based largely on descriptions that 
authors reported and is technical in nature. Exhibit B.1 above contains definitions for some of the terms 
used. Links to each study’s review in the WWC’s Review of Individual Studies database are also 
included in the details below. The Review of Individual Studies database includes details for each 
study, including findings for each outcome that met WWC standards. 


Study details for Alamprese et al. (2011): Making Sense of Decoding and Spelling 

Study overview This study investigated the Making Sense of Decoding and Spelling 
curriculum, which provided explicit instruction on reading and spelling, 
which is direct and purposeful instruction focused on these skills. 


This evidence review considered Alamprese et al. (2011) as two separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparisons between learners who received the 
Making Sense of Decoding and Spelling (MSDS) curriculum and learners 
who did not. The profile for the Alamprese et al. (2011) [K-3 curriculum 
adapted for adult learners] describes the comparisons between learners 
who received the adapted K-3 curriculum and learners who did not. 


Category(ies): Strategy(ies) Literacy instruction: Explicit instruction on reading and spelling, and 
curricular materials 


WWC rating Meets WWC standards with reservations. The study is a cluster 
randomized controlled trial with high individual-level attrition, and the 
groups of learners were shown to be similar at the start of the study. 


Setting The study took place in 23 adult literacy programs located in 12 states. 
These programs included 71 reading classes and 34 instructors. 


Methods This study involved both a cluster randomized controlled trial (RCT) and a 
cluster quasi-experimental design. The RCT component used a lottery to 
assign programs to either use a curriculum (MSDS) designed for adult 
learners or to continue their existing reading instruction. Outcomes for 
learners in these two groups were compared. As part of a quasi- 
experimental cluster design, the authors also compared the learners in 
the programs assigned to use MSDS to learners in adult basic education 
(ABE) programs whose instructors were using a K-3 curriculum adapted 
for ABE learners. 


Study sample The initial sample consisted of 349 learners. These learners attended an 
adult literacy program that provided class-based instruction to English- 
speaking adults at the intermediate level. Sixty-six percent of the learners 
were female. The race/ethnicity distribution of learners was 35 percent 
White, 24 percent Hispanic, 20 percent Black, 15 percent Asian, and 6 
percent in an unspecified other category. Thirty-five percent were born 
and educated outside of the United States. Sixty-three percent had low 
incomes based on the poverty threshold of $12,000 annual salary. Thirty- 
one percent were identified as having a learning disability. 


Treatment group 


The MSDS curriculum was designed specifically for adult learners and 
used to teach decoding and spelling. Instruction lasted approximately 30 
weeks, with classes meeting from one to five days per week. The 
curriculum includes a review of alphabetic decoding skills and principles 
and teaches a strategy for decoding multisyllabic words. Instruction was 
primarily delivered to the whole group in scripted lessons, but lessons 
include paired- and individual-learner reading practice designed to 
improve reading speed. Each lesson includes progress monitoring 
assessments. Instructors were given lesson plans with examples and 
presentation materials. 


Comparison group 


There were two comparison groups. In both groups, classes lasted about 
30 weeks and met one to five days per week. 

The authors refer to the first group as the control condition, because a 
lottery was used to construct this group. Instructors in these programs 
continued to use their existing reading instruction. These classes did not 
use a published scope and sequence. Teachers typically included some 
decoding, but placed more emphasis on spelling, vocabulary, and 
comprehension. 

The second group received explicit instruction on reading and spelling 
that was based on an adapted curriculum. The structured decoding 
curricula were designed for K-3 students, but were adapted for use with 
adults. Additional information about the curricula was not provided. 


Outcomes and measurement 


Study authors reported 11 outcome measures of academic skills that met 
standards: 


e 8inthe alphabetics domain 
— Developmental Spelling Test 
— Test of Word Reading Efficiency: Phonemic Decoding Efficiency 
subtest 
— Test of Word Reading Efficiency: Sight Word Efficiency subtest 
— Wide Range Achievement Test Revision 3: Spelling subtest 
— Wide Range Achievement Test Revision 3: Reading subtest 
— Woodcock-Johnson Tests of Achievement Revised: Word Attack 
subtest 
— Woodcock-Johnson Tests of Achievement Revised: Letter-Word 
Identification subtest 
— Letter-Sound survey 
e 2inthe comprehension domain 
— Nelson Reading Comprehension Test 
— Nelson Word Meaning Test 
e inthe reading fluency domain 
— Passage Reading Test 
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Additional implementation details The researchers measured attendance and hours of instruction for each of 
the study groups. The mean number of hours of instruction was 
approximately 65 among those using a K-3 curriculum adapted for adult 
learners, 60 among those using existing reading instruction, and 50 
among those using the MSDS curriculum. Attendance rates were 57 
percent, 51 percent, and 55 percent, respectively. The researchers also 
looked at whether MSDS was implemented as intended by calculating the 
percentage of lessons taught, the number of hours of study curriculum 
offered to learners, and the level of fidelity to the scripted lessons based 
on classroom observations. The median percentage of lessons taught was 
92 percent; the mean total hours of study curriculum was 27; and the 
mean fidelity scores were 2.18 on a scale of 0 to 3 on the classroom 
observation form, where a score of 3 means all segments were taught as 
scripted. 


Study Review URL in WWC Database _https://ies.ed.gov/ncee/wwe/Study/87298 
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Study details for Alamprese et al. (2011): K-3 curriculum adapted for adult learners 


Study overview This study investigated the use of a K-3 curriculum that was adapted for 
adult learners and provided explicit instruction on reading and spelling. 
This evidence review considered Alamprese et al. (2011) as two separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparisons with learners who received a K-3 
curriculum adapted for adult learners and learners who did not. The 
Alamprese et al. (2011) profile, Making Sense of Decoding and Spelling, 
describes the comparisons with learners who received the Making Sense 
of Decoding and Spelling (MSDS) curriculum with learners who did not. 


Category(ies): Strategy(ies) Literacy instruction: Explicit instruction on reading and spelling 


WWC rating Meets WWC standards with reservations. The study used a cluster 
quasi-experimental design, and the groups of learners were shown to be 
similar at the start of the study. 


Setting The study took place in 23 adult literacy programs located in 12 states. 
These programs included 71 reading classes and 34 instructors. 


Methods This study used a cluster quasi-experimental design. Learners in adult 
education programs who received a K-3 curriculum adapted for adult 
learners were compared with two other sets of learners. The first set of 
learners received their existing reading instruction, and the second set of 
learners received the MSDS curriculum. 


Study sample The initial sample consisted of 349 learners. These learners attended an 
adult literacy program that provided class-based instruction to English- 
speaking adults at the intermediate level. Sixty-six percent of learners 
were female. The race/ethnicity distribution of learners was 35 percent 
White, 24 percent Hispanic, 20 percent Black, 15 percent Asian, and 6 
percent in an unspecified other category. Thirty-five percent were born 
and educated outside of the United States. Sixty-three percent had low 
incomes based on the poverty threshold of $12,000 annual salary. Thirty- 
one percent were identified as having a learning disability. 


Treatment group The treatment curriculum provided explicit instruction on reading and 
spelling. The curriculum was adapted for adult learners from a structured 
curriculum, originally designed for K-3 students, that focused on decoding 
and spelling. Additional information about the curricula was not provided. 
Classes lasted approximately 30 weeks and met one to five days per week. 
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Comparison group 


There were two comparison conditions. In both groups, classes lasted 
approximately 30 weeks and met one to five days per week. 


The first group continued using the existing reading instruction. These 
classes did not use a published scope and sequence. Teachers typically 
included some decoding in their instruction, but placed more emphasis on 
spelling, vocabulary, and comprehension. 

The second comparison group used a curriculum called Making Sense of 
Decoding and Spelling, which was designed specifically for adult learners. 
Teachers in this condition used the study curriculum to teach decoding 
and spelling. The curriculum includes a review of alphabetic decoding 
skills and principles and teaches a strategy for decoding multisyllabic 
words to be used throughout the curriculum. Instruction was primarily 
delivered to the whole group in scripted lessons, but lessons include 
paired- and individual-learner reading practice designed to improve 
reading speed. Each lesson includes progress monitoring assessments. 
Instructors were given lesson plans with examples and presentation 
materials. 


Outcomes and measurement 


Study authors reported nine outcome measures of academic skills that 
met standards: 
e Seven in the alphabetics domain 
— Developmental Spelling Test 
— Test of Word Reading Efficiency: Phonemic Decoding Efficiency 
subtest 
— Wide Range Achievement Test Revision 3: Spelling subtest 
— Wide Range Achievement Test Revision 3 Reading subtest 
— Woodcock-Johnson Tests of Achievement Revised: Word Attack 
subtest 
— Woodcock-Johnson Tests of Achievement Revised: Letter-Word 
Identification subtest 
— Letter-Sound survey 
e One in the comprehension domain 
— Nelson Reading Comprehension Test 


e One in the reading fluency domain 
— Passage Reading Test 


Study authors examined other outcomes that did not meet standards. 


Additional implementation details 


The researchers measured attendance and hours of instruction for each of 
the study groups. The mean number of hours of instruction was 
approximately 65 among those using a K-3 curriculum adapted for adult 
learners, 60 among those using existing reading instruction, and 50 
among those using the MSDS curriculum. Attendance rates were 57 
percent, 51, and 55 percent, respectively. The researchers also looked at 
whether MSDS was implemented as intended by calculating the 
percentage of lessons taught, the number of hours of study curriculum 
offered to learners, and the level of fidelity to the scripted lessons based 
on classroom observations. The median percentage of lessons taught was 
92 percent; the mean total hours of study curriculum was 27; and the 
mean fidelity scores were 2.18 on a scale of 0 to 3 on the classroom 
observation form, where a score of 3 means all segments were taught as 
scripted. 


Study Review URL in WWC Database _hittps://ies.ed.gov/ncee/wwe/Study/88727 
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Study details for Anderson et al. (2017) 


Study overview 


This study investigated Accelerating Opportunity (AO), an integrated 
education and training (IET) co-teaching model that also provided career 
navigation supports and wraparound services to address basic needs and 
logistical barriers. 


Category(ies): Strategy(ies) 


Career development and transition instruction: Integrated education 
and training (IET) using co-teaching approaches 

Individualized and targeted supports for learners: Career navigation 
supports and wraparound services 


WWC rating 


Meets WWC standards with reservations. The study used a quasi- 
experimental design, and the groups of learners were shown to be similar 
at the start of the study. 


Setting 


The study took place in four states; however, the findings that meet WWC 
standards involved only two states. Within these two states, the study took 
place in 25 community colleges or technical schools. 


Methods 


The study used a propensity score matching analysis that matched 
learners in the AO program with non-AO learners drawn from the same 
programs where AO learners were recruited. The two groups of learners 
were matched so they were similar on observed characteristics. 


Study sample 


The initial sample from the analyses of the two states that met standards, 
which were Illinois and Louisiana, consisted of 8,451 learners. These 
learners either participated in the AO program or were enrolled in one of 
three recruitment sources in participating states: adult education, career 
and technical education, or developmental education. The weighted 
analytic sample is 59 percent female, 35 percent White, 45 percent Black, 
and 13 percent Hispanic. Twenty-one percent of the learners (weighted 
percentage) received a Pell grant. 


Treatment group 


Learners in the treatment group participated in an IET co-teaching model 
based on I-BEST called AO. AO provided funding to community and 
technical colleges to develop accessible career pathways in high-demand 
occupations. Career pathways are sequenced programs that allow learners 
to obtain one credential that can lead to others meaningful to the field. 
The AO pathway sequence typically includes 12 credits and is designed to 
be completed in one year or less. The model includes instruction that 
integrates basic skills with career and technical education, and it provides 
wraparound and career navigation services. 


Comparison group 


The learners in the comparison group continued their existing 
instructional practices, which could include being enrolled in 
developmental education or career and technical education programs. 
The study did not include additional details on the comparison condition. 
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Outcomes and measurement 


Study authors reported four outcome measures of employment that met 
standards: 
e Two in the employment short-run domain: Whether the learner was 
employed: 

— In Quarter 1 (Illinois) 

— In Quarter 1 (Louisiana) 
e Two in the employment long-run domain: Whether the learner was 

employed: 

— In Quarter 3 (Louisiana) 

— In Quarter 4 (Illinois) 
Study authors reported four outcome measures of earnings that met 
standards: 
e Two in the earnings short-run domain: 

— Earnings in Quarter 1 (Illinois) 

— Earnings in Quarter 1 (Louisiana) 
e Two in the earnings long-run domain: 

— Earnings in Quarter 3 (Louisiana) 

— Earnings in Quarter 4 (Illinois) 
Study authors examined other outcomes that either did not meet 
standards or that met standards but were considered supplemental 
findings according to the review protocol. Supplemental findings are not 
reported in the evidence review. 


Additional implementation details 


The study does not provide specific information about support for 
implementation. 


Study Review URL in WWC Database _ https://ies.ed.gov/ncee/wwe/Study/88714 
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Study details for Batchelder & Rachal (2000) 


Study overview 


This study investigated the use of a computer-assisted instruction program 
designed to help learners prepare for the high school equivalency test by 
providing instruction in math and English. 


Category(ies): Strategy(ies) 


Literacy instruction: Literacy curricular materials 
Numeracy instruction: Numeracy curricular materials 


Adult secondary education: Preparation for the high school equivalency 
test 


Tools to improve access to instruction: Technology learning tools 


WWC rating 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


Setting 


The study took place in an adult education program in a prison. 


Methods 


The study is a randomized controlled trial. Researchers used a lottery to 
assign learners either to receive the supplemental instruction or to a 
comparison condition. 


Study sample 


The initial sample consisted of 71 learners. These learners lacked a high 
school credential and volunteered for the adult education program at the 
prison. Learners were male, and their average age was 30.5. Seventy-nine 
percent were African American, and 21 percent were Caucasian. Thirty- 
five percent scored below an eighth-grade level on the Test of Adult Basic 
Education. 


Treatment group 


The computer-assisted instruction (CAI) program was a supplement to the 
traditional high school equivalency instruction provided to inmates. 
During one of the instructional hours each day, learners were pulled out 
of the classroom to use a "tutorial/drill and practice" software, which 
included math and English instruction. Learners used the program 
independently in the computer lab with no additional instruction from a 
teacher, although they could ask technical questions about using the 
computers. 


Comparison group 


Members of the comparison group received the existing instructional 
practices offered by the prison to help prepare learners for high school 
equivalency exams. Eighty hours of instruction were offered over a four- 
week period, with four hours of instruction per day. Each day, learners 
received one hour each of math, English, history, and science instruction. 
Instructors taught 10 to 15 learners in each class. 


Outcomes and measurement 


Additional implementation details 


Study authors reported two outcome measures of academic skills that 
met standards: 


e One in the general mathematics achievement domain: Comprehensive 
Adult Student Assessment System Math subtest 

e One in the literacy achievement domain: Comprehensive Adult Student 
Assessment System Reading subtest 

Support staff provided technical assistance to learners using CAI in the 


computer lab. The study did not discuss any other support for 
implementation. 


Study Review URL in WWC Database 


https://ies.ed.gov/ncee/wwc/Study/88717 
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Study details for Condelli et al. (2010) 


Study overview 


This study investigated the Sam and Pat Volume 1 textbook and 
curriculum, which gave English learners instruction in reading, writing, 
listening, and speaking. 


Category(ies): Strategy(ies) 


English language acquisition instruction: English language acquisition 
approaches to reading, writing, listening, and speaking, and curricular 
materials 


WWC rating 


Meets WWC group design standards without reservations. The study 
is a cluster randomized controlled trial with low attrition at the cluster and 
individual levels. 


Setting 


Methods 


The study took place in 10 adult education program sites offering classes 
for English language learners in four states. These programs included 66 
adult English as a Second Language (ESL) classrooms. 


The study is a cluster randomized controlled trial. The authors used a 
lottery to assign pairs of classrooms within sites to either receive the Sam 
and Pat Volume 1 reading textbook or to be in the comparison condition. 
Another lottery assigned learners to classrooms. 


Study sample 


The initial sample consisted of 1,344 learners. These learners were eligible 
for low-literacy-level adult education English as a second language classes 
and enrolled in one of the 10 participating adult education programs. 
Fifty-nine percent of learners were female, and the average age was 40. 
About 14 percent were Asian, Pacific Islander, or Native Hawaiian, or 
identified as another unspecified race; 12 percent were African American; 
and 25 percent were White. Race was not reported for 49 percent of 
learners. The authors reported on outcome measures for the full sample 
as well as for Spanish-speaking learners, which met standards. 


Treatment group 


The Sam and Pat Volume I reading textbook was used in instruction. The 
textbook, which is described by the authors as targeted to the needs of 
English learners with low levels of literacy, is a modification of a 
traditional reading curriculum that was developed for native English 
speakers (the Wilson and Orton-Gillingham reading systems). The 
textbook and curriculum give explicit instruction on reading, writing, 
pronunciation, vocabulary, and grammar, designed for English learners. 
The curriculum is designed to give multiple opportunities for repetition, 
guided practice, and review. The vocabulary and reading passages focus 
words and themes relevant to the learners' lived experience. Each lesson 
is designed to include at least one two-and-a-half-hour session per week of 
pre-reading instruction, and one two-and-a-half hour session per week of 
decoding and reading comprehension instruction. The textbook includes 
22 lessons, which developers expect would take two terms to cover. In this 
study, an average of 13 in-class lessons were covered, ranging from 3 to 22 
lessons. Learners in the treatment group attended, on average, 79 hours 
of instruction. 
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Comparison group 


The comparison group received standard preexisting ESL instruction, 
which focused primarily on learning English. The learners in the 
comparison condition spent more time on English language acquisition 
and functional reading, writing, and math instruction, and less time on 
reading development instruction than the treatment classrooms did. On 
average, learners in the comparison condition attended the ESL class for 
approximately 72 hours. 


Outcomes and measurement 


Additional implementation details 


Study authors reported seven outcome measures of academic skills that 
met standards: 
e Three in the alphabetics domain 

— ETS SARA - Decoding subtest 

— Woodcock Johnson III Letter-Word Identification 

— Woodcock Johnson III Word Attack subtests 


e One in the comprehension domain: Woodcock Johnson III Passage 
Comprehension subtest 

e Three in the English Language Proficiency domain 

— Oral and Written Language Scales 

— Receptive One-Word Picture Vocabulary Test 

— Woodcock Johnson III Picture Vocabulary subtest). 
The outcomes for Spanish-speaking students were considered 
supplemental findings per the review protocol and are not reported in the 
evidence review. The authors also examined other outcomes that did not 
meet standards. 


Treatment teachers received three days of training, which included an 
overview of the approach, specific guidance on lesson planning, and 
opportunities for reflection. The treatment developers provided a 
refresher webinar just before the start of the second term. The trainers 
also conducted a site visit to each treatment classroom about six weeks 
into the beginning of each term and called the teachers biweekly during 
the first two months. They also provided one-day individualized training 
to teachers who were having the most difficulty implementing the 
curriculum as determined by the site visits and phone calls. 


Study Review URL in WWC Database _ hitps://ies.ed.gov/ncee/wwe/Study/87305 
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Study details for Cook et al. (2018) 


Study overview 


This study investigated the Patient Care Pathway (PCP) program, which 
provided contextualized basic instruction, bridge classes, and wraparound 
services to address basic needs and logistical barriers. 


Category(ies): Strategy(ies) 


Career development and transition instruction: Contextualized basic 
skills instruction and bridge classes 


Individualized and targeted supports for learners: Wraparound services 


WWC rating 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


Setting 


The study took place in a two-year college in the Midwest. 


Methods 


The study is a randomized controlled trial. The authors used a lottery to 
assign learners to have access to the PCP program or to a comparison 
condition. 


Study sample 


The initial sample consisted of 500 learners. These learners had scores on 
the Comprehensive Adult Student Assessment System test that fell within 
a range consistent with low skills, but not very poor skills. The eligible 
range of scores varied by program and college. Eighty-four percent of 
learners were female. Sixty-seven percent were White, non-Hispanic; 21 
percent were Black, non-Hispanic; and 9 percent were Hispanic. Twenty- 
three percent of learners were age 20 or younger, 21 percent were ages 21 
to 24, 30 percent were ages 25 to 34, and 25 percent were age 35 or older. 
Forty-seven percent had a high school diploma, a diploma equivalent, or 
less. Learners reported an average income at baseline of $33,165. On 
measures of socioeconomic status, 36 percent reported receiving benefits 
from the Special Supplemental Nutrition Program for Women, Infants, 
and Children (WIC) or the Supplemental Nutrition Assistance Program 
(SNAP); 4 percent reported receiving public assistance or welfare, and 34 
percent reported experiencing financial hardship. 


Treatment group 


The PCP program is a course of study designed to accelerate entry into a 
degree-granting program in a health field. The content of the courses is 
contextualized to health careers, and the program also includes 
wraparound supports, including advising services to help learners enroll 
in courses. 


Comparison group 


The comparison group continued to receive existing courses and supports 
at the college. This included non-contextualized remediation courses and 
access to advisors who were unaffiliated with the PCP academies. 


Outcomes and measurement 


Study authors reported two outcomes of educational progress that met 

standards: 

e One in the progressing in college domain: Hours of occupational 
training at a college 


e One in the industry-recognized credential, certificate, or license (short- 
run) domain: Whether the learner received an occupational or 
educational credential from any source 


Study authors examined other outcomes that either did not meet 
standards or that met standards but were considered supplemental 
findings per the review protocol. Supplemental findings are not reported 
in the evidence review. 
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Additional implementation details The study does not provide specific information about support for 
implementation. 


Study Review URL in WWC Database _https://ies.ed.gov/ncee/wwe/Study/87236 


B-23 


Study details for Glosser et al. (2018) 


Study overview 


This study investigated the Integrated Basic Education and Skills Training 
(I-BEST) program, which provides integrated education and training (IET) 
through co-teaching approaches, co-enrollment with credit courses at 
colleges, career navigation services, and wraparound services to address 
basic needs and logistical barriers. 


Category(ies): Strategy(ies) 


Career development and transition instruction: IET using co-teaching 
approaches and co-enrollment with credit courses at colleges 
Individualized and targeted supports for learners: Career navigation 
supports and wraparound services 


WWC rating 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


Setting 


The study took place in three two-year colleges in the Northwest. 


Methods 


The study is a randomized controlled trial. The authors assigned learners 
through a lottery to receive the I-BEST program or to a comparison 
condition. 


Study sample 


The initial sample consisted of 632 learners. These learners had scores on 
the Comprehensive Adult Student Assessment System test that fell within 
a range consistent with low skills, but not very poor skills. The range of 
scores that determined eligibility varied by program and college. Fifty- 
eight percent of learners were female. Fifty-five percent of learners were 
White, non-Hispanic; 8 percent were Black, non-Hispanic; and 26 percent 
were Hispanic. Twenty-two percent of learners were age 20 or younger, 15 
percent were ages 21 to 24, 30 percent were ages 25 to 34, and 33 percent 
were age 35 or older. Thirty-one percent had less than a high school 
degree, and 40 percent had a high school diploma or the equivalent. 
Learners reported an average income at baseline of $22,110. On measures 
of socioeconomic status, 59 percent reported receiving benefits from the 
Special Supplemental Nutrition Program for Women, Infants, and 
Children (WIC) or the Supplemental Nutrition Assistance Program (SNAP); 
21 percent reported receiving public assistance or welfare, and 49 percent 
reported experiencing financial hardship. 


Treatment group 


The I-BEST program is intended to provide low-skill learners with a career 
pathway through a community or technical college to obtain a credential 
or degree that is valued in a particular occupation (such as automotive, 
electrical, nursing assistant, or welding positions). The programs offered 
courses that provided learners with occupational credit that could lead to 
workforce credentials within one or two quarters. The courses offered 
integrated occupational training and basic skills instruction. Learners 
could also take additional basic skills classes designed to support the 
integrated courses. The program also gave learners financial assistance to 
completely cover tuition, and advisors to support them during 
enrollment, while they were taking courses, and to plan for their career. 


Comparison group 


The comparison group continued to enroll in existing courses and 
receiving typical supports at the colleges. This included non- 
contextualized remediation courses and access to advising services that 
were unaffiliated with the I-BEST program. 
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Outcomes and measurement 


Study authors reported one outcome measure of educational progress 

that met standards: 

e Industry-recognized credential, certificate, or license (short-run) domain: 
Whether the learner received an occupational or educational 
credential from any source 

Study authors reported one outcome measure of employment that met 

standards: 

e Employment (short-run) domain: Whether the learner worked in a job 
requiring at least mid-level skills 


Study authors reported one outcome measure of earnings that met 

standards: 

e Earnings (short-run) domain: Whether the learner worked in a job 
paying $12 per hour or more 

Study authors examined other outcomes that either did not meet 

standards or that met standards but were considered supplemental 

findings per the review protocol. Supplemental findings are not reported 

in the evidence review. 


Additional implementation details 


The study does not provide specific information about support for 
implementation. 


Study Review URL in WWC Database 


https://ies.ed.gov/ncee/wwc/Study/87238 
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Study details for Gray et al. (2018) 


Study overview 


The study investigated the semantic mapping with morpho-phonemic 
analysis (SM-MPA) tutoring program, which provided explicit instruction 
on reading. 


Category(ies): Strategy(ies) 
WWC rating 


Literacy instruction: Explicit instruction on reading 


Meets WWC standards without reservations. The study is a randomized 
controlled trial with low attrition. 


Setting 
Methods 


The study took place in an adult learning center in the Mid-Atlantic. 


The study is a randomized controlled trial. The authors matched learners 
on language learning background and baseline reading assessments, and 
then assigned each learner within the matched pairs, through a lottery, to 
receive semantic mapping with morpho-analysis tutoring, or toa 
comparison condition. 


Study sample 


The initial sample consisted of 46 learners. These learners were enrolled 
in or had recently completed a high school equivalency exam program, 
had basic English proficiency, and were between the ages of 18 and 31, 
among other criteria. The study included adult learners whose average 
age was 24, and who had not received a high school equivalency 
credential. On average, learners had completed schooling through grade 
10. Fifty-six percent were bilingual, 56 percent were female, and 56 
percent were Latino. 


Treatment group 


The SM-MPA tutoring program implemented five elements of evidence- 
based practice: using principles of effective morphological teaching, 
creating word sums, studying morphological relatives, teaching flexible 
syllable segmentation, and assigning primary syllable stress. The SM-MPA 
program focused on teaching 40 academic vocabulary words (derived 
from Latin and Greek word origins and included in a high school civics 
textbook) using a semantic graphic organizer to examine the meaning and 
sounds of the roots and components of each word. Each tutoring session 
took place over a two-hour period each week. The treatment group 
attended programming over a six-week period, with instruction taking 
place in Weeks 2 through 5 (eight hours of individual tutoring). Weeks 1 
and 6 were dedicated to pre- and post-testing. 


Comparison group 


The semantic mapping with whole word study (SM-WWS) tutoring 
program used traditional vocabulary instruction methods to teach the 
same 40 words that were being taught to the treatment group. SM-WWS 
implemented three elements of evidence-based vocabulary instruction: 
information about definitions and sentence contexts, multiple exposure, 
and student engagement. The instruction focused on teaching whole 
words without analyzing words’ internal meaning or sound structures. 
The sessions also took place over a six-week period, with eight hours of 
individual tutoring. 
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Outcomes and measurement 


met standards: 


e Six in the alphabetics domain 


Woodcock Johnson Tests of Achievement III 
Identification subtest 

Woodcock Johnson Tests of Achievement III 
Woodcock Johnson Tests of Achievement III 
Woodcock Johnson Tests of Achievement III 
subtest 

Read Words 

Spell Words 


e Five in the comprehension domain 


Woodcock Johnson Tests of Achievement III 
Comprehension subtest 

Woodcock Johnson Tests of Achievement III 
subtest 

Woodcock Johnson Tests of Achievement III 
subtest 

Match Definitions 

Complete Sentences 


Study authors reported eleven outcome measures of academic skills that 


: Letter-Word 
: Word Attack subtest 


: Spelling subtest 
: Spelling of Sounds 


: Passage 
: Reading Vocabulary 


: Picture Vocabulary 


Study authors examined another outcome that did not meet standards. 


Additional implementation details 


implementation. 


The study does not provide specific information about support for 


Study Review URL in WWC Database _https://ies.ed.gov/ncee/wwe/Study/87308 
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Study details for Greenberg et al. (2011): Decoding and Fluency 


Study overview 


Category(ies): Strategy(ies) 


This study investigated an oral reading approach and the SRA/McGraw Hill 
Direct Instruction Corrective Reading program, which provided explicit 
instruction on reading. 

This evidence review considered Greenberg et al. (2011) as four separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison of learners who received the Decoding 
and Fluency approach with learners who did not. Three other profiles 
examine the Decoding, Comprehension, and Fluency; Decoding, 
Comprehension, Fluency, and Extensive Reading; and the Extensive 
Reading approaches. 


Literacy instruction: Explicit instruction on reading and curricular 
materials 


WWC rating 


Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study. 


Setting 


The study took place in 23 adult literacy programs. 


Methods 


This is a randomized controlled trial. Authors used a lottery to assign 
learners to either receive one of the approaches to teaching literacy or be 
in a comparison group. However, the assignment was compromised 
because the analysis was restricted to learners who were exposed to a set 
number of hours of instruction after random assignment. The authors 
demonstrated that the groups of learners in this restricted sample were 
similar at the beginning of the study. 


Study sample 


The initial study sample consisted of 86 learners. There were 198 learners 
across all four studies. These learners could read at levels that fell between 
the 2nd and 6th grade levels. About 55 percent of learners were African 
American, 28 percent were Hispanic, 12 percent were Asian, and 6 percent 
were White. Sixty-seven percent were female. Fifty-six percent were 
English language learners, and the rest were native English speakers. 
Twenty-nine percent of learners repeated at least one grade of school, 15 
percent attended special education classes while in school, and 49 percent 
reported graduating from high school. 


Treatment group 


The treatment included explicit instruction on reading and curriculum 
using two parts of the SRA/McGraw Hill Direct Instruction Corrective 
Reading program: decoding and fluency. In the Decoding component (D), 
learners learned skills such as phonemic relationships, new sound 
combinations, word endings, and letter and sound combinations. In the 
Fluency component (F), learners independently practiced a passage until 
they improved their number of correct words read per minute by 40 
percent over their initial level. Teachers gave each learner a passage at his 
or her reading level and used a guided repeated oral reading approach. 
Learners practiced for 15 minutes. If they met or exceeded their target, 
they moved on to a new passage. During each two-hour class, time was 
allocated as follows: 100 minutes of decoding, 15 minutes of fluency, and 5 
minutes for a break. The average number of decoding lessons was 75. 
Instruction was provided for two hours per class, four times a week, for 12 
and a half weeks. Learners attending all classes received 100 hours of 
instruction. 
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Comparison group The comparison group used a popular existing adult literacy curriculum. 
Teachers were advised to focus on teaching prime frequency words, W-H 
questions (who, what, when, where, why how), spelling, oral reading, 
journal writing, categorization of words, sentence structure, and 
mechanics of punctuation. Books were not used, but teachers could use 
excerpts based on learners’ interest. Instruction was provided for two 
hours per class, four times a week, for 12 and a half weeks. The curriculum 
did not allocate specific amounts of time for each topic. Learners 
attending all classes received 100 hours of instruction. 


Outcomes and measurement Study authors reported three outcome measures of academic skills that 
met standards: 
e Two in the reading fluency domain 
— Woodcock Johnson III Reading Fluency subtest 
— Gray Oral Reading Test IV Fluency subtest) 
e One in the comprehension domain: Woodcock Johnson Passage 
Comprehension subtest 
Study authors examined other outcomes that did not meet standards. 


Additional implementation details The teachers were hired specifically for the study. They had teaching 
backgrounds, but none had experience with the approaches in the 
SRA/McGraw Hill Direct Instruction Corrective Reading program. Each 
teacher received one week of training in adult literacy awareness and 
sensitivity, along with training on each approach. Once classes were in 
session, coaches observed each teacher five times during the year. 
Teachers received additional training if necessary. 
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Study details for Greenberg et al. (2011): Decoding, Comprehension, and Fluency 


Study overview 


Category(ies): Strategy(ies) 


This study investigated an oral reading approach and the SRA/McGraw Hill 
Direct Instruction Corrective Reading program, which provided explicit 
instruction on reading. 

This evidence review considered Greenberg et al. (2011) as four separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison of learners who received the Decoding, 
Comprehension, and Fluency approach with learners who did not. Three 
other profiles examine the Decoding and Fluency; Decoding, 
Comprehension, Fluency, and Extensive Reading; and Extensive Reading 
approaches. 


Literacy instruction: Explicit instruction on reading and curricular 
materials 


WWC rating 


Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study. 


Setting 


The study took place in 23 adult literacy programs. 


Methods 


This is a randomized controlled trial. Authors used a lottery to assign 
learners to receive one of the approaches to teaching literacy, or toa 
comparison group. However, the assignment was compromised because 
the analysis was restricted to learners who were exposed to a set number 
of hours of instruction after random assignment. The authors 
demonstrated that the groups of learners in this restricted sample were 
similar at the beginning of the study. 


Study sample 


The initial sample consisted of 81 learners. There were 198 learners across 
all four studies. These learners could read at levels between 2nd and 6th 
grade. About 55 percent of learners were African American, 28 percent 
were Hispanic, 12 percent were Asian, and 6 percent were White. Sixty- 
seven percent were female. Fifty-six percent were English language 
learners, and the rest were native English speakers. Twenty-nine percent 
of learners repeated at least one grade of school; 15 percent attended 
special education classes while in school; and 49 percent reported 
graduating from high school. 
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Treatment group 


The treatment included explicit instruction on reading and curriculum 
using three parts of the SRA/McGraw Hill Direct Instruction Corrective 
Reading program: decoding, comprehension, and fluency. In the 
Decoding component (D), learners learned skills such as phonemic 
relationships, new sound combinations, word endings, and letter and 
sound combinations. In the Comprehension component (C), learners 
learned how to organize groups of facts and use analogies, classifications, 
deductions, inductions, descriptions, conclusions, contradictions, and 
written directions. In the Fluency component (F), learners independently 
practiced a passage until they improved their number of correct words 
read per minute by 40 percent over their initial level. Teachers gave each 
learner a passage at his or her reading level and used a guided repeated 
oral reading approach. Learners practiced for 15 minutes. If they met or 
exceeded their target, they moved on to a new passage. During each two- 
hour class, time was allocated as follows: 50 minutes of decoding, 50 
minutes of comprehension, 15 minutes of fluency, and 5 minutes for a 
break. The average number of decoding lessons was 48, and the average 
number of comprehension lessons was 34. Instruction was provided for 
two hours per class, four times a week, for 12 and a half weeks. Learners 
attending all classes received 100 hours of instruction. 


Comparison group 


The comparison group used a popular existing adult literacy curriculum. 
Teachers were advised to focus on teaching prime frequency words, W-H 
questions (who, what, when, where, why how), spelling, oral reading, 
journal writing, categorization of words, sentence structure, and 
mechanics of punctuation. Books were not used, but teachers could use 
excerpts based on learners’ interest. Instruction was provided for two 
hours per class, four times a week, for 12 and a half weeks. The curriculum 
did not allocate specific amounts of time for each topic. Learners 
attending all classes received 100 hours of instruction. 


Outcomes and measurement 


Study authors reported four outcome measures of academic skills that 
met standards: 
e Two in the comprehension domain 
— Woodcock Johnson Passage Comprehension subtest 
— Gray Oral Reading Test IV Comprehension subtest 
e Two in the reading fluency domain 
— Woodcock Johnson III Reading Fluency subtest 
— Gray Oral Reading Test IV Fluency subtest 


Study authors examined other outcomes that did not meet standards. 


Additional implementation details 


The teachers were hired specifically for the study. They had teaching 
backgrounds, but none had experience with the approaches in the 
SRA/McGraw Hill Direct Instruction Corrective Reading program. Each 
teacher received one week of training in adult literacy awareness and 
sensitivity, along with training on each approach. Once classes were in 
session, coaches observed each teacher five times during the year. 
Teachers received additional training if necessary. 
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Study details for Greenberg et al. (2011): Decoding, Comprehension, Fluency, and 


Extensive Reading 


Study overview 


This study investigated extended reading practice and an oral reading 
approach, as well as the use of the SRA/McGraw Hill Direct Instruction 
Corrective Reading program, which provided explicit instruction on 
reading. 

This evidence review considered Greenberg et al. (2011) as four separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison of learners who received the Decoding, 
Comprehension, Fluency, and Extensive Reading approach with learners 
who did not. Three other profiles examine the Decoding and Fluency; 
Decoding, Comprehension, and Fluency; and the Extensive Reading 
approaches. 


Category(ies): Strategy(ies) 


Literacy instruction: Explicit instruction on reading, extended reading 
practice, and curricular materials 


WWC rating 


Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study. 


Setting 
Methods 


The study took place in 23 adult literacy programs. 


This is a randomized controlled trial. Authors assigned learners through a 
lottery to receive one of the approaches to teaching literacy, or toa 
comparison group. However, the assignment was compromised because 
the analysis was restricted to learners who were exposed to a set number 
of hours of instruction after random assignment. The authors 
demonstrated that the groups of learners in this restricted sample were 
similar at the beginning of the study. 


Study sample 


The initial sample consisted of 89 learners. There were 198 learners across 
all four studies. These learners could read between the 2nd and 6th grade 
levels. About 55 percent of learners were African American, 28 percent 
were Hispanic, 12 percent were Asian, and 6 percent were White. Sixty- 
seven percent were female. Fifty-six percent were English language 
learners, and the rest of were native English speakers. Twenty-nine 
percent of learners repeated at least one grade of school, 15 percent 
attended special education classes while in school, and 49 percent 
reported graduating from high school. 
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Treatment group 


The treatment included explicit instruction on reading, extended practice, 
and curriculum using four parts of the SRA/McGraw Hill Direct Instruction 
Corrective Reading program: decoding, comprehension, extensive 
reading, and fluency. In the Decoding component (D), learners learned 
skills such as phonemic relationships, new sound combinations, word 
endings, and letter and sound combinations. In the Comprehension 
component (C), learners learned how to organize groups of facts and use 
analogies, classifications, deductions, inductions, descriptions, 
conclusions, contradictions, and written directions. Extensive Reading (E) 
is extended reading practice using an extensive library of high interest/low 
vocabulary books. Learners engaged in sustained silent reading, an 
activity where the teacher read aloud and the learner followed along, 
followed by discussion about the books everyone was reading. In the 
Fluency component (F), learners independently practiced a passage until 
they improved their number of correct words read per minute by 40 
percent over their initial level. Teachers gave each learner a passage at his 
or her reading level and used a guided repeated oral reading approach. 
Learners practiced for 15 minutes. If they met or exceeded their target, 
they moved on to a new passage. During each two-hour class, time was 
allocated as follows: 33 minutes of decoding; 33 minutes of 
comprehension; 33 minutes of extensive reading (18 minutes of sustained 
silent reading, 10 minutes of teacher read-aloud, and 5 minutes of book 
discussion); 15 minutes of fluency; and 5 minutes for a break. The average 
number of decoding lessons was 36, and the average number of 
comprehension lessons was 31. Instruction was provided for two hours 
per class, four times a week, for 12 and a half weeks. Learners attending all 
classes received 100 hours of instruction. 


Comparison group 


The comparison group used a popular existing adult literacy curriculum. 
Teachers were advised to focus on teaching prime frequency words, W-H 
questions (who, what, when, where, why how), spelling, oral reading, 
journal writing, categorization of words, sentence structure, and 
mechanics of punctuation. Books were not used, but teachers could use 
excerpts based on learners’ interest. Instruction was provided for two 
hours per class, four times a week, for 12 and a half weeks. The curriculum 
did not allocate specific amounts of time for each topic. Learners 
attending all classes received 100 hours of instruction. 


Outcomes and measurement 


Study authors reported three outcome measures of academic skills that 
met standards: 
e One in the comprehension domain: Woodcock Johnson Passage 
Comprehension subtest 
e Two in the reading fluency domain 
— Woodcock Johnson III Reading Fluency subtest 
— Gray Oral Reading Test IV Fluency subtest 


Study authors examined other outcomes that did not meet standards. 


B-33 


Additional implementation details The teachers were hired specifically for the study. They had teaching 
backgrounds, but none had experience with the approaches in the 
SRA/McGraw Hill Direct Instruction Corrective Reading program. Each 
teacher received one week of training in adult literacy awareness and 
sensitivity, along with training on each approach. Once classes were in 
session, coaches observed each teacher five times during the year. 
Teachers received additional training if necessary. 


Study Review URL in WWC Database _hittps://ies.ed.gov/ncee/wwe/Study/88772 
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Study details for Greenberg et al. (2011): Extensive Reading 


Study overview 


This study investigated the use of extended reading practice. 


This evidence review considered Greenberg et al. (2011) as four separate 
studies that examined four different approaches to teaching literacy. This 
profile describes the comparison of learners who received the Extensive 
Reading approach with learners who did not. Three other profiles 
examine the Decoding and Fluency; Decoding, Comprehension, and 
Fluency; and Decoding, Comprehension, Fluency, and Extensive Reading 
approaches. 


Category(ies): Strategy(ies) 


Literacy instruction: Extended reading practice 


WWC rating 


Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study. 


Setting 


The study took place in 23 adult literacy programs. 


Methods 


This is a randomized controlled trial. Authors used a lottery to assign 
learners to either receive one of the approaches to teaching literacy or be 
in a comparison group. However, the assignment was compromised 
because the analysis was restricted to learners who were exposed to a set 
number of hours of instruction after random assignment. The authors 
demonstrated that the groups of learners in this restricted sample were 
similar at the beginning of the study. 


Study sample 


The initial sample consisted of 62 learners. There were 198 learners across 
all four studies. These learners could read between the 2nd and 6th grade 
levels. About 55 percent of learners were African American, 28 percent 
were Hispanic, 12 percent were Asian, and 6 percent were White. Sixty- 
seven percent were female. Fifty-six percent were English language 
learners, and the rest were native English speakers. Twenty-nine percent 
of learners repeated at least one grade of school; 15 percent attended 
special education classes while in school; and 49 percent reported 
graduating from high school. 


Treatment group 


Extensive Reading (ER) is extended reading practice using an extensive 
library of high interest/low vocabulary books. Learners engaged in 
sustained silent reading—an activity in which the teacher read aloud, and 
the learner followed along, with a discussion afterward about the books 
everyone was reading. During each class, time was allocated as follows: 
two blocks of silent sustained reading of 40 minutes each, 15 minutes of 
teacher read-aloud activities, 10 minutes of book discussion, and a 5- 
minute break. 


Comparison group 


The comparison group used a popular existing adult literacy curriculum. 
Teachers were advised to focus on teaching prime frequency words, W-H 
questions (who, what, when, where, why how), spelling, oral reading, 
journal writing, categorization of words, sentence structure, and 
mechanics of punctuation. Books were not used, but teachers could use 
excerpts based on learners’ interest. Instruction was provided for two 
hours per class, four times a week, for 12 and a half weeks. The curriculum 
did not allocate specific amounts of time for each topic. Learners 
attending all classes received 100 hours of instruction. 
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Outcomes and measurement Study authors reported three outcome measures of academic skills that 
met standards: 


e One in the alphabetics domain: Woodcock Johnson Work Attack 
subtest 


e One in the comprehension domain: Woodcock Johnson Passage 
Comprehension subtest 


e One in the reading fluency domain: Gray Oral Reading Test IV Fluency 
subtest 


Study authors examined other outcomes that did not meet standards. 


Additional implementation details The teachers were hired specifically for the study. They had teaching 
backgrounds, but none had experience with the approaches in the 
SRA/McGraw Hill Direct Instruction Corrective Reading program. Each 
teacher received one week of training in adult literacy awareness and 
sensitivity, along with training on each approach. Once classes were in 
session, coaches observed each teacher five times during the year. 
Teachers received additional training if necessary. 
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Study details for Hamadyk & Zeidenberg (2018) 


Study overview 


This study investigated the Workforce Training Academy Connect (WTA 
Connect) program, which provided technology learning tools, coordinated 
enrollment in adult education and occupational skills training, workforce 
preparation, career navigation supports, and wraparound services to 
address basic needs and logistical barriers. 


Category(ies): Strategy(ies) 


Tools to improve access to instruction: Technology learning tools 

Career development and transition instruction: Coordinated enrollment 
in adult education and occupational skills training, and workforce 
preparation 

Individualized and targeted supports for learners: Career navigation 
supports and wraparound services 


WWC rating 


Setting 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


The study took place in a community college in the Midwest. 


Methods 


The study is a randomized controlled trial. The authors used a lottery to 
assign learners to either the WTA Connect program or to a comparison 
condition. 


Study sample 


The initial sample consisted of 943 learners. These learners had math and 
reading skills between the 6th and 8th grade level. Sixty-three percent of 
learners were female. Thirty-four percent were White non-Hispanic, 47 
percent were Black non-Hispanic, and 15 percent were Hispanic. Fourteen 
percent of learners were age 20 or younger, 16 percent were ages 21 to 24, 
28 percent were ages 25 to 34, and 42 percent were age 35 or older. Forty 
percent had less than a high school degree, and 37 percent had a high 
school diploma or the equivalent. Learners reported an average income at 
the start of the study of $16,364. On measures of socioeconomic status, 66 
percent reported receiving benefits from the Special Supplemental 
Nutrition Program for Women, Infants, and Children (WIC) or the 
Supplemental Nutrition Assistance Program (SNAP); 14 percent reported 
receiving public assistance or welfare; and 63 percent reported 
experiencing financial hardship. 


Treatment group 


Comparison group 


WTA Connect gives low-skill learners an opportunity to enroll in 
occupational certificate courses that they would otherwise be ineligible to 
take. Learners had access to basic skills remediation through online 
courses, advisors who provided support during enrollment and monitored 
academic progress, and employment assistance resources upon 
completion of an occupational certificate course. Learners took the 
remediation and occupational training courses tuition-free and could 
receive transportation assistance and screening for public benefit 
eligibility. 

The comparison group continued to use existing education, training, and 
support services available in the community outside of the WTA Connect 
program. Additionally, learners could access training and services at the 
college or from other sources in the community. Some of these services, 
such as screening for benefits eligibility, were like services offered by WTA 
Connect. 
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Outcomes and measurement 


Study authors reported two outcome measures of educational progress 
that met standards: 


e One in the progressing in college domain: Hours of college occupational 
training 

e One in the industry-recognized credential, certificate, or license (short- 
run) domain: Whether the learner received an occupational or 
educational credential from any source 


Study authors reported one outcome measure of employment that met 
standards: 


e Employment (short-run): Whether the learner was working in a job 
requiring at least mid-level skills 


Study authors reported one outcome measure of earnings that met 
standards: 


e Earnings (short-run): Whether the learner was working in a job paying 
$12 per hour or more 

Study authors examined other outcomes that either did not meet 

standards or that met standards but were considered supplemental 

findings per the review protocol. Supplemental findings are not reported 

in the evidence review. 


Additional implementation details 


The study does not provide specific information about support for 
implementation. 


Study Review URL in WWC Database _https://ies.ed.gov/ncee/wwe/Study/87237 


B-38 


Study details for Hock & Mellard (2011): Prediction Strategy 


Study overview 


This study investigated the Prediction Strategy, which provided explicit 
instruction on reading. 

This evidence review considered Hock & Mellard (2011) as four separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison of learners who received the Prediction 
Strategy with learners who did not. The Hock & Mellard (2011) 
[Summarization Strategy] profile describes the comparison of learners 
who received the Summarization Strategy with learners who did not. 
There are no profiles for the two other instructional models (Bridging 
Strategy and Building Fluency Strategy) because these studies did not 
meet standards. 


Category(ies): Strategy(ies) 


WWC rating 


Literacy instruction: Explicit instruction on reading 


Meets WWC standards with reservations. The study is a randomized 
controlled trial with high attrition, and the groups of learners were shown 
to be similar at the start of the study. 


Setting 


Methods 


The study took place in three adult education programs operated by 
community colleges in the Midwest. These programs included 29 reading 
classes. 


The study is a randomized controlled trial. The authors used a lottery to 
assign learners to receive the Prediction Strategy instructional model or to 
comparison classes within adult education programs. 


Study sample 


The initial sample consisted of 143 learners. There were 375 learners 
across all four studies. These learners were at least 16 years old, had 
withdrawn from secondary education without earning a credential or 
without attaining 8th grade equivalency in reading, writing, or math skills; 
and were U.S. citizens or authorized to work in the United States. Across 
the four studies, 63 percent of learners were female, and 33 percent were 
White. On average, learners had completed a level of education between 
9th and 10th grade. 


Treatment group 


The program taught learners to use the “CLUE” mnemonic: C - check for 
clues; L - link to prior knowledge; U - unveil predictions, and E - examine 
the reading. Before reading a passage, a learner “checks for clues” by 
visually scanning the reading materials. The next step involves the 
learners thinking about what the clues tell them and linking them to what 
they already know. In the third step, the learner makes and unveils a 
prediction about the passage’s main topic. In general, classes were 
planned for eight weeks, four days per week, and 50-60 minutes per day. 


Comparison group 


Comparison group instructors continued using their existing teaching 
practices. They delivered materials to prepare learners for the high school 
equivalency exam using whole group instruction, small groups, and 
individual tutoring. Comparison classes were also planned for eight 
weeks, four days per week, and 50-60 minutes per day. 
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Outcomes and measurement 


Study authors reported two outcome measures of academic skills that 
met standards: 


e Comprehension domain 
— Woodcock Reading Mastery Test-Revised Passage Comprehension 
subtest 
— Gray Oral Reading Tests Fourth Edition Comprehension subtest 


Additional implementation details 


Research staff trained instructors in the treatment condition before the 
study began and gave them reading materials for adult learners. To 
monitor implementation, researchers collected instructors’ records of 
attendance and minutes of instruction and made audio- and video- 
recordings of all class. The authors indicated that the differences between 
treatment and comparison groups in the number of minutes of instruction 
or learner attendance rates were not large enough to be distinguished 
from statistical noise. They also indicated that the treatment instructors 
taught the “CLUE” mnemonic strategy and that the comparison 
instructors did not. 
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Study details for Hock & Mellard: 2011: Summarization 


Study overview 


Category(ies): Strategy(ies) 


This study investigated the Summarization Strategy, which provided 
explicit instruction on reading. 

This evidence review considered Hock & Mellard (2011) as four separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison of learners who received the 
Summarization Strategy with learners who did not. The Hock & Mellard, 
2011 [Prediction Strategy] profile describes the comparison of learners 
who received the Prediction Strategy with learners who did not. There are 
no profiles for the two other instructional models (Bridging Strategy and 
Building Fluency Strategy) because these studies did not meet standards. 


Literacy instruction: Explicit instruction on reading 


WWC rating 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


Setting 


The study took place in four adult education programs operated by a 
community-based organization in the Midwest. These programs included 
28 reading classes. 


Methods 


The study is a randomized controlled trial. The authors assigned learners 
through a lottery to receive the Summarization Strategy instructional 
model or to comparison classes within adult education programs. 


Study sample 


The initial sample consisted of 146 learners. There were 375 learners 
across all four studies. These learners were at least 16 years old, had 
withdrawn from secondary education without earning a credential or 
without attaining 8th grade equivalency in reading, writing, or math skills; 
and were U.S. citizens or authorized to work in the United States. Across 
the four studies, 63 percent of learners were female, and 33 percent were 
White. On average, learners had completed a level of education between 
9th or 10th grade. 


Treatment group 


The program taught learners to use the “READ” mnemonic: R - review the 
passage; E - evaluate the paragraph by asking themselves a question about 
what they are going to read; A - answer the question with a paraphrase; D 
- determine how to summarize a passage. In the first step, learners are 
taught to look for clues, and make predictions about the entire passage. 
After making a prediction, they recursively find main ideas and important 
details, and paraphrase small chunks of text, like paragraphs. Finally, they 
pull together all the information into a passage or document summary. 
Steps R and D are completed for the whole passage, and steps E and A are 
completed for each paragraph or chunk of text. In general, classes were 
planned for eight weeks, four days per week, and 50-60 minutes per day. 


Comparison group 


Comparison group instructors continued to use their existing teaching 
practices. They delivered materials to prepare learners for the high school 
equivalency exam using whole group instruction, small groups, and 
individual tutoring. Comparison classes were also planned for eight 
weeks, four days per week, and 50-60 minutes per day. 
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Outcomes and measurement 


Study authors reported two outcome measures of academic skills that 
met standards: 


e Comprehension domain 
— Woodcock Reading Mastery Test-Revised Passage Comprehension 
subtest 
— Gray Oral Reading Tests Fourth Edition Comprehension subtest 


Additional implementation details 


Research staff trained instructors in the treatment condition before the 
study began and gave them reading materials for adult learners. To 
monitor implementation, researchers collected instructors’ records of 
attendance and minutes of instruction and made audio- and video- 
recordings of all class sessions. The authors indicated that the differences 
between treatment and comparison groups in the number of minutes of 
instruction or learner attendance rates were not large enough to be 
distinguished from statistical noise. They also indicated that the treatment 
instructors taught the “READ” mnemonic strategy, and the comparison 
instructors did not. 


Study Review URL in WWC Database _hittps://ies.ed.gov/ncee/wwe/Study/88734 
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Study details for Martin & Broadus (2013) 


Study overview 


This study investigated a general educational development (GED) bridge 
program that provided contextualized basic skills instruction, which 
helped learners prepare for the high school equivalency, and provided 
career navigation supports. 


Category(ies): Strategy(ies) 


Adult secondary education: Preparation for the high school equivalency 
test 

Career development and transition instruction: Contextualized basic 
skills instruction and bridge classes 

Individualized and targeted supports for learners: Career navigation 
supports 


WWC rating 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


Setting 


The study took place in a community college in the Mid-Atlantic. 


Methods 


Study sample 


The study is a randomized controlled trial. The authors used a lottery to 
assign learners to either receive the GED bridge program or be ina 
comparison group. 

The initial sample consisted of 276 learners. These learners scored at a 7th 
grade reading level or higher, were age 18 or older, and had an income 
below 200 percent of the federal poverty level. Sixty-seven percent of 
learners were female; the average age of learners was about 27. Just over 
50 percent were Hispanic or Latino, about 35 percent were non- 
Hispanic/Latino African American, and 15 percent were of other races or 
ethnicities. About 53 percent were receiving public assistance, and 38 
percent were employed when randomly assigned. Twenty-five percent 
scored at the 7th or 8th grade reading levels, whereas 16 percent scored at 
the 9th grade level, and 34 percent scored at the 10th grade level or 
higher. About 15 percent attained a 9th grade education or lower, whereas 
30 percent finished 10th grade, 36 percent finished 11th grade, and 9 
percent finished 12th grade; 10 percent did not report the grade level they 
had attained. 


Treatment group 


The GED Bridge to Health and Business program is a curriculum preparing 
learners for the high school equivalency exam, and it also promotes 
general academic habits that will help learners succeed in later college 
classes or training programs. The program uses a "contextualized 
curriculum" that aligns instruction to one of two professional focuses: 
health or business. The program is structured and delivered in a fashion 
similar to a regular college course, with the aim of familiarizing learners 
with this format so they are more prepared for college. Instruction was 
provided for 108 hours over 12 weeks. Learners also receive one-on-one 
guidance to help explore career and educational options. GED Bridge 
instructors were full-time, master’s-level educators trained in adult 
literacy instruction and contextualized curriculum development. 
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Comparison group The comparison group continued to use existing high school equivalency 
exam preparation practices in a program called GED Prep offered at the 
college. Instruction was provided for 60 hours over nine weeks, and 
materials included only the high school equivalency exam textbook. No 
counseling or support were provided. GED Prep instructors were part- 
time adjunct instructors with some background in adult education, but 
they were not trained in adult literacy. 


Outcomes and measurement Study authors reported three outcome measures of educational 

progress that met standards: 

e One in the college enrollment domain: Whether the learner ever 
enrolled at a City University of New York (CUNY) community college 

e One in the progressing in college domain: Whether the learner ever 
enrolled at CUNY for a second semester 

e One in the completing high school (short-run) domain: Whether the 
learner passed the GED exam 


Additional implementation details The study does not provide specific information about support for 
implementation. 


Study Review URL in WWC Database _ hittps://ies.ed.gov/ncee/wwe/Study/87318 
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Study details for Miller et al. (2016) 


Study overview 


Category(ies): Strategy(ies) 


This study investigated the YouthBuild model, which provided 
preparation for the high school equivalency test, integrated education and 
training (IET) through co-planning and alternative teaching, and 
wraparound services to address basic needs and logistical barriers. 


Adult secondary education: Preparation for the high school equivalency 
test 


Career development and transition instruction: IET through co-planning 
and alternative teaching 


Individualized and targeted supports for learners: Wraparound services 


WWC rating 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


Setting 


The study took place in 75 YouthBuild programs across the country. 
Programs were located in 29 states, the U.S. Virgin Islands, and 
Washington DC, and were found both in both densely populated urban 
centers and rural areas. 


Methods 


The study is a randomized controlled trial. The authors used a lottery to 
assign learners to either receive the YouthBuild program or be ina 
comparison group. 


Study sample 


The initial sample consisted of 3,929 learners. These learners generally 
had to be between the ages of 16 and 24, have withdrawn from secondary 
education without a credential, have a referral from a high school, or have 
a deficiency in basic skills. Eligibility criteria varied depending on the 
YouthBuild program site. Sixty-three percent of learners were Black, and 
64 percent were male. Ninety percent had not graduated from high school 
or passed a high school equivalency exam, and 79 percent were under the 
age of 22. The authors reported on outcome measures for the full sample 
as well as by gender and educational attainment subgroups, which met 
standards. 


Treatment group 


Comparison group 


The YouthBuild program includes instruction in basic skills, remedial 
education, and alternative education that is designed to lead to a high 
school diploma or equivalent credential. It also offers vocational training, 
which is typically training in construction, as well as youth development 
services including leadership training and community service. Supportive 
wraparound services could include counseling, case management, life- 
skills training, workforce preparation, follow-up services, stipends, and 
help with child care, transportation or housing. 


For two years, members of the comparison group were prohibited from 
enrolling in YouthBuild programs that were participating in the study and 
given information about other appropriate services in the community. 
About 1 percent of learners in the comparison group enrolled in local 
YouthBuild programs that were not participating in the study. 
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Outcomes and measurement 


Study authors reported six outcomes of educational progress that met 
standards: 


e Four in the college enrollment domain 
— Whether the learner enrolled in a four-year institution at 30 
months 
— Whether the learner enrolled in a two-year institution at 30 
months 
— Whether the learner enrolled in an institution offering less than 
two years of higher education at 30 months 
— Whether the learner enrolled in any college at 48 months 
e One in the college degree attainment (short-run) domain: Whether the 
learner received a degree at 30 months 


e One in the college degree attainment (long-run) domain: Whether the 
learner received a degree at 48 months 

Study authors reported two outcomes of employment that met 

standards: 

e One in the employment (short-run) domain: Whether the learner was 
employed in Year 1 

e One in the employment (long-run) domain: Whether the learner was 
employed in Year 4 

Study authors reported two outcomes of earnings that met standards: 

e One in the earnings (short-run) domain: Earnings in Year 1 

e One in the earning (long-run) domain: Earnings in Year 4 


The outcomes for subgroups defined by gender and educational 
attainment were considered supplemental findings per the review 
protocol and are not reported in the evidence review. The authors also 
examined other outcomes that did not meet standards. 


Additional implementation details 


The study does not provide specific information about support for 
implementation. 


Study Review URL in WWC Database _hitps://ies.ed.gov/ncee/wwe/Study/87321 
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Study details for Modicamore et al. (2017) 


Study overview 


This study investigated the Accelerating Connections to Employment 
(ACE) program, which provided integrated education and training (IET) 
through co-teaching approaches, co-enrollment with credit courses at 
colleges, career navigation supports, and wraparound services to address 
basic needs and logistical barriers. 


Category(ies): Strategy(ies) 


Career development and transition instruction: IET through co-teaching 
approaches and co-enrollment with credit courses at colleges 
Individualized and targeted supports for learners: Career navigation 
supports and wraparound services 


WWC rating 


Meets WWC standards without reservations. The study is a 
randomized controlled trial with low attrition. 


Setting 


The study took place in nine Workforce Investment Board sites in four 
states. 


Methods 


The study is a randomized trial. The authors assigned learners through a 
lottery to receive the Accelerating Connections to Employment program 
or to a comparison condition within site. 


Study sample 


The initial sample consisted of 2,168 learners. These learners had to 
possess basic skills, including basic language proficiency, among other 
eligibility criteria. Minimum and maximum cut scores on skills 
assessments varied by Workforce Investment Board site. Learner 
characteristics varied by state. The percentage of learners who were White 
ranged from 1.5 to 54 percent. The percentage of learners who were Black 
ranged from 36 to 96 percent. The percentage of learners who were 
Latino ranged from 1.5 to 40 percent. Between 52 and 92 percent of 
learners were female, and between 4.6 and 58 percent of learners had less 
than a high school diploma. 


Treatment group 


The ACE program is in part based on Washington State’s Integrated Basic 
Education and Skills Training model, and incorporates basic skills, 
occupational skills, and job readiness training. Key features of the ACE 
model included IET delivered through co-teaching, and co-enrollment 
with credit courses that allowed learners to earn credentials, career 
navigation, and support services (including academic and transportation 
support). 


Comparison group 


The comparison group had access to the existing training and services 
available for workers in each state. Compared with the ACE program, 
existing services were less likely to offer advanced career services. 
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Outcomes and measurement 


Study authors reported one outcome measure of educational progress 
that met standards: 


e Industry-recognized credential, certificate, or license (short-run) 
domain: Whether the learner holds a vocational, technical, or 
professional certificate or license 

Study authors reported one outcome measure of employment that met 

standards: 

e Employment (short-run) domain: Whether the learner was employed in 

Year 1, for all states 

Study authors reported three outcome measures of earnings that met 

standards: 

e Earnings (short-run) domain 
— Earnings in Year 1 for Maryland and Texas 
— Earnings in Year 1 for Atlanta 
— Earnings in Year 1 for Connecticut 

Study authors examined other outcomes that did not meet standards. 


Additional implementation details 


The study does not provide specific information about support for 
implementation. 


Study Review URL in WWC Database _ https://ies.ed.gov/ncee/wwe/Study/88740 
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Study details for Robinson (2018) 


Study overview 


This study investigated the Pure and Complete Phonics curriculum, which 
provided explicit instruction on reading. 


Category(ies): Strategy(ies) 


WWC rating 


Literacy instruction: Explicit instruction on reading and curricular 
materials 


Meets WWC standards without reservations. It is a randomized 
controlled trial with low attrition. 


Setting 


The study took place in six correctional institutions in the Midwest. 


Methods 


This study is a randomized trial. Administrative staff assigned learners 
through a lottery to receive the Pure and Complete Phonics curriculum or 
to a comparison condition within correctional institutions. 


Study sample 


The initial sample consisted of 44 learners. These learners attended a 
school program within the participating correctional facilities with at least 
six months remaining until their release and scored at a fifth grade or 
lower reading level. The average age of learners in the treatment group 
was 42.1, and the average age of those in the comparison group was 36.5. 
No percentages on race or ethnicity were given, but different learners 
identified as Black, Asian, Hispanic, or White. 


Treatment group 


The program was a modification of the Orton-Gillingham curriculum 
known as Pure and Complete Phonics (PCP). PCP is an instructional 
technique that uses direct, explicit, and multisensory instruction on 
reading. PCP is scripted and sequenced. It employs specific references and 
formats to use the 26 alphabet letters and 103 phonemes or phonemic 
units to identify spelling and reading words. Learners received instruction 
using the PCP curriculum Monday through Friday for one hour each day 
for 15 weeks. 


Comparison group 


The comparison group continued to receive instruction based on their 
institution’s existing reading curriculum for one hour per day (Monday 
through Friday) for 15 weeks. The existing curriculum is not named or 
described. 


Outcomes and measurement 


Study authors reported five outcome measures of academic skills that 
met standards: 


e Four in the alphabetics domain 
— Woodcock Johnson Tests of Achievement III: Letter-Word 
Identification subtest 
— Woodcock Johnson Tests of Achievement III: Work Attack subtest 
— Woodcock Johnson Tests of Achievement III: Spelling subtest 
— Woodcock Johnson Tests of Achievement III: Spelling of sounds 
subtest 


e One in the reading fluency domain: Woodcock Johnson Tests of 
Achievement III: Reading Fluency subtest 
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Additional implementation details The study author led a two-day training workshop for teachers before the 
study began. The training included an overview of the PCP curriculum, 
instruction on the PCP formats, and opportunities to practice the PCP 
approach with each other. The training emphasized the curriculum’s 
instructional approach, including the importance of following the 
curriculum script. The trainer modeled each of the steps in the scripts and 
had teachers practice teaching each other. All teachers had an Adult Basic 
Education certification and had, on average, 17 years of teaching 
experience. 


Study Review URL in WWC Database _https://ies.ed.gov/ncee/wwe/Study/87326 
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Study details for Sabatini et al. (2011): Corrective Reading 


Study overview This study investigated an adaptation of the Corrective Reading (CR) 
approach, which provided explicit instruction on reading. 
This evidence review considered Sabatini et al. (2011) as three separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison with learners who received an 
adaptation of the Corrective Reading Approach with learners who did not. 
Two other profiles describe the Guided Repeated Reading approach, 
designed specifically for adults, and a modified version of the Retrieval, 
Automaticity, Vocabulary Elaboration-Orthography approach. 


Category(ies): Strategy(ies) Literacy instruction: Explicit instruction on reading and curricular 
materials 
WWC rating Meets WWC standards with reservations. The study is a compromised 


randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study. 


Setting The study took place in several large adult education centers in two major 
cities in the mid-Atlantic and southern regions of the United States. 


Methods This is a randomized controlled trial. The authors assigned learners 
through a lottery to receive one of the approaches to teaching literacy. 
However, the assignment was compromised because some learners 
dropped out before completing the treatment and were replaced with 
learners from the participating classrooms, assuming scheduling and tutor 
availability. The authors demonstrated that the final group of learners 
were similar to each other at the beginning of the study. 


Study sample The initial sample consisted of 300 learners. These learners scored below 
the 7th grade level on a word recognition test and demonstrated English 
proficiency if they were not native English speakers. Sixty-seven percent 
of learners were female; on average, they were 36 years old. Eighty-three 
percent were African American, 9 percent were Latino(a), and 8 percent 
were White. 


Treatment group The adaptation of the CR program provided explicit instruction on 
reading, using a traditional phonics instruction for treating reading 
disabilities that is commonly used in adolescents. Instruction focuses on 
strengthening and expanding the reader’s mastery of grapheme-phoneme 
correspondences, and on word recognition. Through CR, learners are 
taught the structure of words through an explicit, systematic, and 
sequenced curriculum that teaches decoding and spelling, with phonemic 
analyses that are taught in relation to syllable types. Learners progress 
from a phonological focus to word-level practice, and eventually to 
processing words quickly by recognizing patterns and reading context. 
Learners also read controlled (decodable) texts to gain fluency. 
Instructional time is 80 to 90 percent phonics, and 10 to 20 percent 
fluency. Instructional sessions were conducted three times per week for 
10-18 weeks. The goal was to complete 45 sessions of 75 minutes each. 


Comparison group The comparison group participated in one of two tutoring programs: 
either Retrieval, Automaticity, Vocabulary Elaboration-Orthography 
(RAVE-O), or Guided Repeated Reading (GRR). These programs are 
described in their own profiles. 
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Outcomes and measurement Study authors reported six outcome measures of academic skills that 
met standards: 


e Four in the alphabetics domain 
— Woodcock Johnson: Word Attack subtest 
— Woodcock Johnson: Letter-word identification subtest 
— Test of Word Reading Efficiency: Sight Word Efficiency subtest 
— Test of Word Reading Efficiency: Phonemic Decoding Efficiency 
subtest 
e One in the comprehension domain: Woodcock Johnson Passage 
Comprehension subtest 


e One in the reading fluency domain: Woodcock Johnson: Reading 
Fluency subtest 


Additional implementation details Tutors had a bachelor’s degree and were comfortable with technology. 
Tutor training included a one-day (5-6 hour) workshop, two individual 
follow-up meetings of about 1-2 hours each with experienced tutors or 
trainers, and review and practice with materials, role-plays, and reviews 
of sample sessions. Total time spent training was 12 to 18 hours. During 
implementation, conferences were conducted to ensure techniques were 
being consistently applied. 


Study Review URL in WWC Database _hittps://ies.ed.gov/ncee/wwe/Study/87327 
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Study details for Sabatini et al. (2011): Guided Repeated Reading 


Study overview 


Category(ies): Strategy(ies) 


This study investigated a guided repeated reading approach, which 
provided explicit instruction on reading. 

This evidence review considered Sabatini et al. (2011) as three separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison with learners who received a Guided 
Repeated Reading program designed specifically for adults with learners 
who did not. Two other profiles describe an adaptation of Corrective 
Reading Approach and a modified version of Retrieval, Automaticity, 
Vocabulary Elaboration-Orthography approaches. 


Literacy instruction: Explicit instruction on reading 


WWC rating 


Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study. 


Setting 


The study took place in several large adult education centers in two major 
cities in the mid-Atlantic and southern regions of the United States. 


Methods 


This is a randomized controlled trial. The authors assigned learners 
through a lottery to receive one of the approaches to teaching literacy. 
However, the assignment was compromised because some learners 
dropped out before completing the treatment and were replaced with 
learners from the participating classrooms, assuming scheduling and tutor 
availability. The authors demonstrated that the final group of learners 
were similar to each other at the beginning of the study. 


Study sample 


The initial sample consisted of 300 learners. These learners scored below 
the 7th grade level on a word recognition test and demonstrated English 
proficiency if they were not a native English speaker. Sixty-seven percent 
of learners were female; on average, they were 36 years old. Eighty-three 
percent were African American, 9 percent were Latino(a), and 8 percent 
were White. 


Treatment group 


The Guided Repeated Reading (GRR) program provided explicit 
instruction on reading and was designed specifically for adult learners. It 
targets text fluency skills, although phonics instruction is also embedded 
within the GRR approach. Instruction includes teacher modeling oral 
reading, shared reading between learner and teacher-reading orally in 
unison, and learners reading orally by themselves up to three times in the 
same session. The reading passages are brief, contain predictable and 
rhythmic text to promote fluency, and are selected based on the level and 
interests of the adult learners. Less than 10 to 20 percent of instructional 
time is spent on phonics, and 80 to 90 percent or more is spent on 
fluency. Instructional sessions were conducted three times per week for 
10-18 weeks. The goal was to complete 45 sessions of 75 minutes each. 


Comparison group 


The comparison group participated in one of two tutoring programs: 
either Retrieval, Automaticity, Vocabulary Elaboration-Orthography 
(RAVE-O), or Corrective Reading (CR). These programs are described in 
their own profiles. 
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Outcomes and measurement Study authors reported six outcome measures of academic skills that 
met standards: 


e Four in the alphabetics domain 
— Woodcock Johnson: Word Attack subtest 
— Woodcock Johnson: Letter-word identification subtest 
— Test of Word Reading Efficiency: Sight Word Efficiency subtest 
— Test of Word Reading Efficiency: Phonemic Decoding Efficiency 
subtest 
e One in the comprehension domain: Woodcock Johnson Passage 
Comprehension subtest 


e One in the reading fluency domain: Woodcock Johnson: Reading 
Fluency subtest). 


Additional implementation details Tutors had a bachelor’s degree and were comfortable with technology. 
Tutor training included a one-day (5-6 hour) workshop, two individual 
follow-up meetings of about 1-2 hours each with experienced tutors or 
trainers, and review and practice with materials, role-plays, and reviews 
of sample sessions. Total time spent training was 12 to 18 hours. During 
implementation, conferences were conducted to ensure techniques were 
being consistently applied. 


Study Review URL in WWC Database _https://ies.ed.gov/ncee/wwe/Study/88780 
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Study details for Sabatini et al. (2011): Retrieval, Automaticity, Vocabulary 


Exploration - Orthography 


Study overview 


Category(ies): Strategy(ies) 


This study investigated the Retrieval, Automaticity, Vocabulary 
Elaboration-Orthography (RAVE-O) approach, which provided explicit 
instruction on reading. 

This evidence review considered Sabatini et al. (2011) as three separate 
studies that examined different approaches to teaching literacy. This 
profile describes the comparison with learners who received a modified 
version of Retrieval, Automaticity, Vocabulary Elaboration-Orthography 
approach with learners who did not. Two other profiles describe an 
adaptation of Corrective Reading Approach and a Guided Repeated 
Reading program designed specifically for adults. 

Literacy instruction: Explicit instruction on reading and curricular 
materials 


WWC rating 


Meets WWC standards with reservations. The study is a compromised 
randomized controlled trial, and the groups of learners were shown to be 
similar at the start of the study. 


Setting 


The study took place in several large adult education centers in two major 
cities in the mid-Atlantic and southern regions of the United States. 


Methods 


This is a randomized controlled trial. The authors used a lottery to assign 
learners to receive one of the approaches to teaching literacy. However, 
the assignment was compromised because some learners dropped out 
before completing the treatment; any learners who dropped out were 
replaced with another learner from the participating classrooms assuming 
scheduling and tutor availability. The authors demonstrated that the final 
group of learners were similar to each other at the beginning of the study. 


Study sample 


The initial sample consisted of 300 learners. These learners scored below 
the 7th grade level on a word recognition test and demonstrated English 
proficiency if they were not a native English speaker. Sixty-seven percent 
of learners were female; on average, they were 36 years old. Eighty-three 
percent were African American, 9 percent were Latino(a), and 8 percent 
were White. 


Treatment group 


The Retrieval, Automaticity, Vocabulary Elaboration-Orthography (RAVE- 
O) program provided explicit instruction on reading by supplementing 
phonics instruction with fluency training and a stronger focus on fluency. 
This approach was based on the Double Deficit hypothesis, which suggests 
deficits in either phonological processing or naming speed can impede 
reading acquisition. RAVE-O is designed to address a naming speed deficit 
or a double deficit and is combined with an abbreviated version of 
Corrective Reading, a systematic phonics program described in its own 
profile. Instructional time is 25 to 35 percent phonics and 65 to 75 percent 
fluency. Instructional sessions were conducted three times per week for 10 
to 18 weeks. The goal was to complete 45 sessions of 75 minutes each. 


Comparison group 


The comparison group participated in one of two tutoring programs: 
either Corrective Reading (CR) or Guided Repeated Reading (GRR). These 
programs are described in their own profiles. 
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Outcomes and measurement Study authors reported six outcome measures of academic skills that 
met standards: 


e Four in the alphabetics domain 
— Woodcock Johnson: Word Attack subtest 
— Woodcock Johnson: Letter-Word Identification subtest 
— Test of Word Reading Efficiency: Sight Word Efficiency subtest 
— Test of Word Reading Efficiency: Phonemic Decoding Efficiency 
subtest 


e One in the comprehension domain: Woodcock Johnson Passage 
Comprehension subtest 

e One in the reading fluency domain: Woodcock Johnson: Reading 
Fluency subtest 


Additional implementation details Tutors had a bachelor’s degree and were comfortable with technology. 
Tutor training included a one-day (5-6 hour) workshop, two individual 
follow-up meetings of about 1-2 hours each with experienced tutors or 
trainers, and review and practice with materials, role-plays, and reviews 
of sample sessions. Total time spent training was 12 to 18 hours. During 
implementation, conferences were conducted to ensure techniques were 
being consistently applied. 


Study Review URL in WWC Database _https://ies.ed.gov/ncee/wwe/Study/88769 
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B.3. Number and List of Effectiveness Studies within Particular Categories of Adult 
Education Strategies, by Outcome Area 
This table expands on Exhibit 3 in the Snapshot to include the specific studies, of the 22 that met What Works Clearinghouse version 4.0 


standards, that fall under each category of adult education strategies. The table includes a short reference for the main study; the full set of 
references can be found in Section B.1. 


Table B.1. Number and List of Effectiveness Studies within Particular Categories of Adult Education Strategies, by Outcome 
Area 


Adult Education Outcome Area 


a 


Category of Adult Education Strategies (Number of studies) Academic Educational 
RYT ACL AY) skills progress Employment Earnings 


Literacy instruction (14) 
Alamprese et al., 2011 [Making Sense of Decoding and Spelling] 
Alamprese et al., 2011 [K-3 curriculum adapted for adult learners] 
Batchelder & Rachal, 2000 
Gray et al., 2018 
Greenberg et al., 2011 [Decoding and Fluency] 
Greenberg et al., 2011 [Decoding, Comprehension, and Fluency] 
Greenberg et al., 2011 [Decoding, Comprehension, Extensive Reading, and Fluency] 
Greenberg et al., 2011 [Extensive reading] 
Hock & Mellard, 2011 [Prediction Strategy] 
Hock & Mellard, 2011 [Summarization Strategy] 
Robinson, 2018 
Sabatini et al., 2011 [Corrective Reading] 
Sabatini et al., 2011 [Guided Repeated Reading] 
Sabatini et al., 2011 [RAVE-O] 
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Adult Education Outcome Area 


oO Tle o 


Category of Adult Education Strategies (Number of studies) Academic Educational 
RY TORU CLAY) skills progress Employment Earnings 


Numeracy instruction (1) 
Batchelder & Rachal, 2000 


English language acquisition instruction (1) 
Condelli et al., 2010 


D- D 


Adult secondary education (3) 
Batchelder & Rachal, 2000 
Martin & Broadus, 2013 
Miller et al., 2016 


fi 


Tools to improve access to adult education instruction (2) 
Batchelder & Rachal, 2000 
Hamadyk & Zeidenberg, 2018 


[i - 


Career development and transition instruction (7) 
Anderson et al., 2017 
Cook et al., 2018 
Glosser et al., 2018 
Hamadyk & Zeidenberg, 2018 
Martin & Broadus, 2013 
Miller et al., 2016 
Modicamore et al., 2017 
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Adult Education Outcome Area 


oO Tl eo 


Category of Adult Education Strategies (Number of studies) Academic Educational 


Study(ies) skills progress Employment Earnings 


Individualized and targeted learner supports (7) 
Anderson et al., 2017 

Cook et al., 2018 

Glosser et al., 2018 

Hamadyk & Zeidenberg, 2018 

Martin & Broadus, 2013 

Miller et al., 2016 

Modicamore et al., 2017 


C_] No rigorous C1 1 study ES 2 to 4 studies M9 5 to 9 studies 8 More than 10 studies 

effectiveness studies 
Source: The 22 studies that met What Works Clearinghouse version 4.0 standards with and without reservations. 
Note: Adult education categories are not mutually exclusive, so a single study may have examined strategies in multiple categories, and some studies examined effects in two 
or more outcome areas. Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding— 
academic skills, educational progress, employment, and earnings. See Appendix A for additional information on how outcomes were grouped into these four adult 
education outcome areas. 
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B.4. Strategies Examined by the 22 Studies that Met Standards and their Effects on Learner 
Outcomes 


For the 22 studies that met What Works Clearinghouse version 4.0 standards, this section contains the strategies they examined and their 
effects on learner outcomes. The evidence review snapshot presents findings for four key adult education outcome areas: academic skills, 
educational progress, employment and earnings. The table below presents findings for these outcome areas, broken out into the outcome 
domains specified within the WWC’s Review of Individual Studies Protocol. The table includes a short reference only for the main study 
reference, although information was also drawn from additional references where available and the full set of references can be found in 
Section B.1. 


Table B.2. The 22 studies that met standards categorized by strategy, and their effect on learner outcomes 


Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Primary reference 
Study 

WWC rating 

¢ Strategies examined 


Reading fluency 
English language 
proficiency 
COimee(solaeel | 
College enrollment 
College degree 
attainment (short-run) 
C Of0) | (fevemre (9 vere 
attainment (long-run) 
Employment 
(short-run) 
Employment 


qi 
oa 
=| q 
tN 
£ £ 
[ay Qa 
2 | 
= io) 

iS) 


General mathematics 
Completing high school 


Literacy achievement 
Progressing in college 


Alamprese et al., 2011: Effects of a structured decoding curriculum on adult literacy learners’ reading development 


Earnings 


Making Sense of Decoding 

and Spelling 

Meets standards with 

reservations © © © 

e Explicit instruction on 
reading and spelling, and 
curricular materials 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Primary reference 
Study 


WWC rating 
¢ Strategies examined 


Reading fluency 
English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 
Earnings 


< 
g | 8 
Te = 
Vv i.) 
fe) x= 
P- z 
[ay Qa 
= = 
< [-) 

O 


General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Literacy achievement 


K-3 curriculum adapted for 
adult learners 


Meets standards with © 8) © 

reservations 

e Explicit instruction on 
reading and spelling 


Anderson et al., 2017: New evidence on integrated career pathways: Final impact report for Accelerating Opportunity 


Meets standards with 
reservations 


e Integrated education and 
training (IET) using co- & +) eS) +) 
teaching approaches, 
career navigation 


supports, and 
wraparound services 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Earnings 


Primary reference 
Study 

WWC rating 

¢ Strategies examined 


Reading fluency 
English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 


| 
g | ¢ 
= q 
Vv i.) 
fe) f= 
P- z 
[ay Q 
= = 
< -) 

O 


Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Batchelder & Rachal, 2000: Efficacy of a computer-assisted instruction program in a prison setting: An experimental study 


Meets standards without 

reservations 

e Literacy curricular 
materials, numeracy 
curricular materials, © © 
preparation for the high 
school equivalency test, 


and technology learning 
tools 


Condelli et al., 2010: The impact of a reading intervention for low-literate adult ESL learners 


Meets standards without 
reservations 


e English language 
acquisition approaches to © © © 
reading, writing, listening, 


and speaking, and 
curricular materials 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Earnings 


Primary reference 
Study 

WWC rating 

¢ Strategies examined 


Reading fluency 
English language 
proficiency 
Credential 
(short-run) 
College enrollment 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 


| 
g | ¢ 
= q 
Vv i.) 
fe) f= 
P- z 
[ay Q 
= = 
< >) 

O 


General mathematics 
Progressing in college 


Literacy achievement 
Completing high school 


Cook et al., 2018: Madison Area Technical College Patient Care Pathway program: Implementation and early impact report 


Meets standards without 
reservations 


e Contextualized basic skills © © 


instruction, bridge classes, 
and wraparound services 


Glosser et al., 2018: Washington State’s Integrated Basic Education and Skills Training (I-BEST) program in three colleges: Implementation and 
early impact report 


Meets standards without 
reservations 


e JET using co-teaching 
approaches, co- +) © © 
enrollment with credit 
courses at colleges, career 
navigation services, and 
wraparound services 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Primary reference 
Study 


WWC rating 
¢ Strategies examined 


English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
(long-run) 
Earnings 
(short-run) 
Earnings 


Reading fluency 


| 
g | ¢ 
Te q 
Vv i.) 
fe) f= 
P- z 
[ay Q 
= = 
< -) 

O 


Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Gray et al., 2018: Morpho-phonemic analysis boosts word reading for adult struggling readers 


Meets standards without 


reservations © © 
e Explicit instruction on 


reading 


Greenberg et al., 2011: A randomized control study of instructional approaches for struggling adult readers 


Decoding and fluency 
Meets standards with 


reservations © © 
e Explicit instruction on 


reading and curricular 
materials 


Decoding, comprehension, 
and fluency 


Meets standards with 


reservations © © 


e Explicit instruction on 
reading and curricular 
materials 


B-64 


Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Primary reference 
Study 


WWC rating 
¢ Strategies examined 


English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 
Earnings 


Reading fluency 
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Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Decoding, comprehension, 

fluency, and extensive 

reading 

Meets standards with 

reservations © © 

e Explicit instruction on 
reading, extended reading 
practice, and curricular 
materials 


Extensive reading 
Meets standards with (—) © © 
reservations 


e Extended reading practice 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Earnings 
(long-run) 


Primary reference 
Study 

WWC rating 

¢ Strategies examined 


Reading fluency 
English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 
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Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Hamadyk & Zeidenberg, 2018: Des Moines Area Community College Workforce Training Academy Connect Program: Implementation and early 
impact report 


Meets standards without 

reservations 

e Technology learning tools, 
coordinated enrollment in 
adult education and © +) © © 
occupational skills 
training, workforce 
preparation, career 
navigation supports, and 
wraparound services 


Hock & Mellard, 2011: Efficacy of learning strategies instruction in adult education 


Prediction strategy 
Meets standards with 


reservations © 


e Explicit instruction on 
reading 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Primary reference 
Study 


WWC rating 
¢ Strategies examined 


Reading fluency 
English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 
Earnings 
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Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Summarization strategy 
Meets standards without 


reservations © 


e Explicit instruction on 
reading 


Martin & Broadus, 2013: Enhancing GED instruction to prepare students for college and careers: Early success in LaGuardia Community College’s 
Bridge to Health and Business Program 


Meets standards without 
reservations 


e Preparation for the high 
school equivalency test, +) +) +) 
contextualized basic skills 
instruction, bridge classes, 


and career navigation 
supports 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Earnings 


Primary reference 
Study 

WWC rating 

¢ Strategies examined 


Reading fluency 
English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 
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Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Miller et al., 2016: Building a future: Interim impact findings from the YouthBuild evaluation 


Meets standards without 
reservations 


e Preparation for the high 
school equivalency test, +) 


IET through co-planning 
and alternative teaching, 
and wraparound services 


oO 
Qo 
© 
© 
0 
© 


Modicamore et al., 2017: Accelerating connections to employment. Volume I: Final evaluation report 


Meets standards without 
reservations 


e JET through co-teaching 
approaches, co- +) +) +) 
enrollment with credit 
courses at colleges, career 


navigation supports, and 
wraparound services 
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Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Primary reference 
Study 


WWC rating 
¢ Strategies examined 


English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
(long-run) 
Earnings 
(short-run) 
Earnings 


Reading fluency 
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Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


Robinson, 2018: A study designed to increase the literacy skills of incarcerated adults 


Meets standards without 

reservations 

e Explicit instruction on © © 
reading and curricular 
materials 


Sabatini et al., 2011: Relative effectiveness of reading intervention programs for adults with low literacy 


Corrective reading 
Meets standards with 


reservations 8) © © 
e Explicit instruction on 


reading and curricular 
materials 

Guided reading 

Meets standards with 


reservations © © © 


e Explicit instruction on 
reading 


B-69 


Adult Education Outcome Areas and RISP Outcome Domains 


Academic skills Educational progress Employment 


Primary reference 
Study 


WWC rating 
¢ Strategies examined 


English language 
proficiency 
Credential 
(short-run) 
(short-run) 
College degree 
attainment (short-run) 
College degree 
attainment (long-run) 
Employment 
(short-run) 
Employment 
Earnings 
(short-run) 
Earnings 


Reading fluency 
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Literacy achievement 
General mathematics 
Progressing in college 
College enrollment 
Completing high school 


RAVE-O 
Meets standards with 


reservations © © © 
e Explicit instruction on 


reading and curricular 
materials 
The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect (—) Negative effect +) Mixed effect 


Source: The 22 studies that met What Works Clearinghouse version 4.0 standards. 
Note: Mixed effects indicates the findings within an outcome domain include both statistically significant positive and statistically significant negative effects. See Appendix A 
for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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B.5. Presence of Rigorous Research on Particular Strategies in Adult Education and the 
Effects on Learner Outcomes 


This section contains a table for each of the seven adult education categories included in the framework used to organize strategies for the 
evidence review. The tables contain information about the specific strategies examined across the 22 students that met What Works 
Clearinghouse version 4.0 standards, as well as the effect found of those strategies on academic skills, educational progress, employment, and 
earnings. While the tables include a short reference for the main study reference, information was also drawn from additional references 
where available and the full reference can be found in Section B.1. 


Table B.3. Presence of rigorous research on selected literacy strategies and the effects on learner outcomes 


Adult Education Outcome Area 


@ B® 


Strategies Academic Educational 


ex 


Study skills progress Employment Earnings 


Explicit instruction on reading 


Explicit instruction on reading alone 
Gray et al., 2018; 
Hock & Mellard, 2011 [Prediction Strategy]; © 
Hock & Mellard, 2011 [summarization Strategy]; 
Sabatini et al., 2011 [Guided Repeated Reading] 


With instruction on spelling 
Alamprese et al., 2011 [K-3 curriculum adapted for adult learners] 


With instruction on spelling and curricular materials 
Alamprese et al., 2011 [Making Sense of Decoding and Spelling] 


oO |O 
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Adult Education Outcome Area 


OT © o 


Strategies Academic Educational 
Study skills progress Employment Earnings 


With curricular materials 

Greenberg et al., 2011 [Decoding and Fluency]; 
Greenberg et al., 2011 [Decoding, Comprehension, and Fluency]; © 
Robinson, 2018; 

Sabatini et al., 2011 [Corrective Reading]; 
Sabatini et al., 2011 [RAVE-O] 


With extended reading practice and curricular materials 
Greenberg et al., 2011 [Decoding, Comprehension, Extensive Reading, and Fluency] 


Greenberg et al., 2011 [Extensive reading] 


Curricular materials 


With numeracy curricular materials; preparation for high school equivalence 
test; and technology learning tools 


Batchelder & Rachal, 2000 


Explicit framing 


Extended reading practice ‘—) 


Explicit writing instruction 


Integrated reading and writing instruction 


Real-world contexts in instruction 


The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect '—) Negative effect ® Mixed effect 


Source: The 14 studies that met What Works Clearinghouse standards and examined literacy strategies in adult education. 

Note: Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding—academic skills, educational 
progress, employment, and earnings. Mixed effects indicates the findings within an outcome area include both statistically significant positive and statistically significant 
negative effects. See Appendix A for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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Table B.4. Presence of rigorous research on selected numeracy strategies and the effects on learner outcomes 


Adult Education Outcome Areas 


7 Tl © 


Strategies Academic Educational 
Study skills progress Employment Earnings 


Curricular materials 


With literacy curricular materials; preparation for high school equivalency test; 
and technology learning tools © 
Batchelder & Rachal, 2000 


Constructivist teaching 


Instruction focused on multiple strands of mathematical proficiency 


Real-world contexts in instruction 


Mathematical practices, including problem solving and mathematical justification 
The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect (—) Negative effect e Mixed effect 


Source: The one study that met What Works Clearinghouse standards and examined numeracy strategies in adult education. 


Note: Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding—academic skills, educational 
progress, employment, and earnings. Mixed effects indicates the findings within an outcome area include both statistically significant positive and statistically significant 
negative effects. See Appendix A for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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Table B.5. Presence of rigorous research on selected English language acquisition strategies and the effects on learner 
outcomes 


Adult Education Outcome Areas 


7 Fl @® « 


Strategies Academic Educational 
Study skills progress Employment Earnings 


Focus on listening and speaking 


With focus on reading and writing; and curricular materials © 
Condelli et al., 2010 


Integrate civics education into English language acquisition instruction 


Real-world contexts in instruction 


Focus on functional language skills needed for the workplace 


Connect with native language literacy skills 
The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect '—) Negative effect e Mixed effect 


Source: The one study that met What Works Clearinghouse standards and examined English language acquisition strategies in adult education. 


Note: Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding—academic skills, educational 
progress, employment, and earnings. Mixed effects indicates the findings within an outcome area include both statistically significant positive and statistically significant 
negative effects. See Appendix A for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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Table B.6. Presence of rigorous research on selected adult secondary education strategies and the effects on learner outcomes 


Adult Education Outcome Areas 


oO Tl we o 


Strategies Academic Educational 
Study skills progress Employment Earnings 


Preparation for high school (HS) equivalency test 


With literacy curricular materials; numeracy curricular materials; and technology 


learning tools © 


Batchelder & Rachal, 2000 

With bridge classes; contextualized basic instruction; and career navigation 
supports 

Martin & Broadus, 2013 


+ 


With integrated education and training (IET) through co-planning and alternate 
teaching; and wraparound services 


Miller et al., 2016 


Competency-based HS equivalency 


@ 
© 
0 


Curricular materials 


Real-world contexts in instruction 
The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect ‘—) Negative effect e Mixed effect 


Source: The three studies that met What Works Clearinghouse standards and examined adult secondary education strategies in adult education. 


Note: Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding—academic skills, educational 
progress, employment, and earnings. Mixed effects indicates the findings within an outcome area include both statistically significant positive and statistically significant 
negative effects. See Appendix A for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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Table B.7. Presence of rigorous research on selected strategies for improving access to instruction and the effects on learner 
outcomes 


Adult Education Outcome Areas 


2 8 » 


Strategies Academic Educational 
Study skills progress Employment Earnings 


Technology learning tools 


With literacy curricular materials; numeracy curricular materials; and preparation 
for high school (HS) equivalency test © 
Batchelder & Rachal, 2000 

With coordinated enrollment in adult education and occupational skills training; 
workforce preparation; career navigation supports; and wraparound services +) © 8) 
Hamadyk & Zeidenberg, 2018 


Distance learning 


Blended learning 


Mobile or online learning tools 


Self-paced in-person facilitated learning 


Alternative locations and times 


Intake and orientation models 


Attendance policies 


Employer partnerships 


The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect (—) Negative effect e Mixed effect 


Source: The two studies that met What Works Clearinghouse standards and examined strategies for improving access to instruction. 


Note: Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding—academic skills, educational 
progress, employment, and earnings. Mixed effects indicates the findings within an outcome area include both statistically significant positive and statistically significant 
negative effects. See Appendix A for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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Table B.8. Presence of rigorous research on selected career development and transition strategies and the effects on learner 
outcomes 


Adult Education Outcome Areas 


Strategies Academic Educational 
Study skills progress Employment Earnings 


Integrated education and training (IET) through co-teaching 


With career navigation supports and wraparound services 
Anderson et al., 2017 


With co-enrollment with credit courses at colleges; career navigation supports; and 
wraparound services 


Glosser et al., 2018 

With co-enrollment with credit courses at colleges; career navigation supports; and 
wraparound services 

Modicamore et al., 2017 


IET through co-planning and alternate teaching 


With preparation for high school (HS) equivalency test and wraparound services 
Miller et al., 2016 


Coordinated enrollment in adult education and occupational skills training 


With technology learning tools; workforce preparation; career navigation supports; 
and wraparound services 


Hamadyk & Zeidenberg, 2018 


o Go 08 @ 
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Adult Education Outcome Areas 


OT © & 


Strategies Academic Educational 
Study skills progress Employment Earnings 


On-ramp and bridge classes focused on preparing learners for occupational skills 
training or college readiness 


With contextualized basic skills instruction and wraparound services © 
Cook et al., 2018 

With preparation for HS equivalency test; contextualized basic skills instruction; and 

career navigation supports +) 
Martin & Broadus, 2013 


Pre-apprenticeships 


Alternative credentialing or badges 


Co-enrollment with developmental education at colleges 
The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect (—) Negative effect e Mixed effect 


Source: The seven studies that met What Works Clearinghouse standards and examined career development and transition strategies. 

Note: Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding—academic skills, educational 
progress, employment, and earnings. Mixed effects indicates the findings within an outcome area include both statistically significant positive and statistically significant 
negative effects. See Appendix A for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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Table B.9. 


Strategies 
Study 


Career navigation supports 


With IET through co-teaching and wraparound services to address basic needs and 
logistical barriers 
Anderson et al., 2017 


Presence of rigorous research on selected targeted learner support strategies and the effects on learner outcomes 


Adult Education Outcome Areas 


Oo Tl w® « 


Academic Educational 
skills progress Employment Earnings 


With IET through co-teaching; co-enrollment with credit courses at colleges; and 
wraparound services to address basic needs and logistical barriers 


Glosser et al., 2018 


With technology learning tools; coordinated enrollment in adult education and 
occupational skills; workforce preparation; and wraparound services to address 
basic needs and logistical barriers 


Hamadyk & Zeidenberg, 2018 


With preparation for high school (HS) equivalency test; contextualized basic 
instruction; and bridge classes 
Martin & Broadus, 2013 


With IET through co-teaching; co-enrollment with credit courses at colleges; and 
wraparound services to address basic needs and logistical barriers 


Modicamore et al., 2017 


Wraparound services to address basic needs and logistical barriers 


With IET through co-planning and alternate teaching; and preparation for HS 
equivalency test 
Miller et al., 2016 


With contextualized basic skills instruction; and bridge classes 
Cook et al., 2018 


oOo |09 oO 08 @ 
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Adult Education Outcome Areas 


Oo le o 


Strategies Academic Educational 
Study skills progress Employment Earnings 


Texting or calls to encourage persistence 


Re-entry initiatives and post-release services for formerly incarcerated individuals 


The following icons indicate presence or lack of a significant effect: 


© No effect +) Positive effect (—) Negative effect e Mixed effect 


Source: The seven studies that met What Works Clearinghouse standards and examined targeted learner supports. 


Note: Outcomes were combined into outcome areas that align with the primary indicators of performance for programs receiving Title II funding—academic skills, educational 
progress, employment, and earnings. Mixed effects indicates the findings within an outcome area include both statistically significant positive and statistically significant 
negative effects. See Appendix A for additional information on how findings were classified as no effect, positive effect, negative effect, and mixed effects. 
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